K-Nearest Neighbors:KNN | Program7 III Bsc Computer Science| M.S University | Web Academy
Program 7
Import necessary modules
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris
import numpy as np
import matplotlib.pyplot as plt
irisData = load_iris()
Create feature and target arrays
X = irisData.data
y = irisData.target
Split into training and test set
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size = 0.2, random_state=42)
neighbors = np.arange(1, 9)
train_accuracy = np.empty(len(neighbors))
test_accuracy = np.empty(len(neighbors))
knn = KNeighborsClassifier(n_neighbors=7)
knn.fit(X_train, y_train)
Calculate the accuracy of the model
print("Accuracy:",knn.score(X_test, y_test))
Loop over K values
for i, k in enumerate(neighbors):
knn = KNeighborsClassifier(n_neighbors=k)
knn.fit(X_train, y_train)
Compute training and test data accuracy
train_accuracy[i] = knn.score(X_train, y_train)
test_accuracy[i] = knn.score(X_test, y_test)
Generate plot
plt.plot(neighbors, test_accuracy, label = 'Testing dataset Accuracy')
plt.plot(neighbors, train_accuracy, label = 'Training dataset Accuracy')
plt.legend()
plt.xlabel('n_neighbors')
plt.ylabel('Accuracy')
plt.show()
***************************************************The End*****************
It is a supervised machine learning algorithm. The algorithm can be used to solve both classification and regression problems.The main distinction here is that classification is used for discrete values, whereas regression is used with continuous ones.
kNN is also called a lazy learning algorithm.
K-NN is a lazy learner because it doesn’t learn a discriminative function from the training data but memorizes the training dataset instead. There is no training time in K-NN. The prediction step in K-NN is expensive. Each time we want to make a prediction, K-NN is searching for the nearest neighbors in the entire training set. An eager learner has a model fitting or training step. A lazy learner does not have a training phase.
Why is the k-nearest neighbors algorithm called “lazy”?
Because it does no training at all when you supply the training data. At training time, all it is doing is storing the complete data set but it does not do any calculations at this point. Neither does it try to derive a more compact model from the data which it could use for scoring. Therefore, we call this algorithm lazy.
Applications of KNN
It is commonly used for simple recommendation systems, pattern recognition, data mining, financial market predictions, intrusion detection, and more.
Recommendation systems: Since KNN can help find users of similar characteristics, it can be used in recommendation systems. For example, it can be used in an online video streaming platform to suggest content a user is more likely to watch by analyzing what similar users watch.
Computer vision: The KNN algorithm is used for image classification. Since it’s capable of grouping similar data points, for example, grouping cats together and dogs in a different class, it’s useful in several computer vision applications.
Finance: It is used to determine the credit-worthiness of a loan applicant. highlights its use in stock market forecasting, currency exchange rates, trading futures, and money laundering analyses.
Healthcare: KNN has also had application within the healthcare industry, making predictions on the risk of heart attacks and prostate cancer. The algorithm works by calculating the most likely gene expressions.
Pattern Recognition: KNN has also assisted in identifying patterns, such as in text and digit classification (link resides outside of ibm.com).
Stock price prediction: Since the KNN algorithm has a flair for predicting the values of unknown entities, it's useful in predicting the future value of stocks based on historical data.
Working of KNN Algorithm
K-nearest neighbors (KNN) algorithm uses ‘feature similarity’ to predict the values of new datapoints which further means that the new data point will be assigned a value based on how closely it matches the points in the training set. We can understand its working with the help of following steps -
Step 1 - For implementing any algorithm, we need dataset. So during the first step of KNN, we must load the training as well as test data.
Step 2 - Next, we need to choose the value of K i.e. the nearest data points. K can be any integer.
Step 3 - For each point in the test data do the following -
3.1 - Calculate the distance between test data and each row of training data with the help of any of the method namely: Euclidean, Manhattan or Hamming distance. The most commonly used method to calculate distance is Euclidean.
3.2 - Now, based on the distance value, sort them in ascending order.
3.3 - Next, it will choose the top K rows from the sorted array.
3.4 - Now, it will assign a class to the test point based on most frequent class of these rows.
Step 4 - End