K-Nearest Neighbors (KNN) Algorithm Explained with Python (2026)

KNN is one of the simplest and most intuitive machine learning algorithms. It makes no distribution assumptions, requires no training phase, and handles both classification and regression. This guide covers everything you need to use KNN effectively.

How KNN Works

KNN is a lazy learner — it memorises the training data and makes predictions at query time. For a new data point: compute distance to every training point, find k nearest neighbours, return majority class (classification) or average value (regression). There is no training phase — everything happens at prediction time.

Distance Metrics

Euclidean (default): sqrt of sum of squared differences. Best for continuous features on similar scales.

Manhattan: sum of absolute differences. More robust to outliers in individual dimensions.

Minkowski: generalises both (p=2 gives Euclidean, p=1 gives Manhattan).

Hamming: fraction of positions differing — for categorical or binary data.

KNN from Scratch

import numpy as np
from collections import Counter

class KNNClassifier:
    def __init__(self, k=5):
        self.k = k

    def fit(self, X, y):
        self.X_train = np.array(X)
        self.y_train = np.array(y)
        return self

    def _distance(self, x1, x2):
        return np.sqrt(np.sum((x1 - x2)**2))

    def predict_single(self, x):
        distances = [self._distance(x, xt) for xt in self.X_train]
        k_indices = np.argsort(distances)[:self.k]
        k_labels  = self.y_train[k_indices]
        return Counter(k_labels).most_common(1)[0][0]

    def predict(self, X):
        return np.array([self.predict_single(x) for x in X])

KNN with Scikit-learn

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

pipe = Pipeline([('scaler', StandardScaler()), ('knn', KNeighborsClassifier(n_neighbors=5))])
pipe.fit(X_train, y_train)
print(f'Accuracy: {pipe.score(X_test, y_test):.4f}')

Finding the Best K

import matplotlib.pyplot as plt
from sklearn.model_selection import cross_val_score

k_range = range(1, 31)
cv_scores = []
for k in k_range:
    knn_cv = Pipeline([('sc', StandardScaler()), ('knn', KNeighborsClassifier(n_neighbors=k))])
    cv_scores.append(cross_val_score(knn_cv, X, y, cv=5, scoring='accuracy').mean())

best_k = list(k_range)[np.argmax(cv_scores)]
print(f'Best k={best_k}  CV Accuracy={max(cv_scores):.4f}')

plt.plot(k_range, cv_scores, 'bo-')
plt.axvline(best_k, color='red', linestyle='--', label=f'Best k={best_k}')
plt.xlabel('k'); plt.ylabel('CV Accuracy'); plt.legend(); plt.show()

KNN for Regression

from sklearn.neighbors import KNeighborsRegressor
from sklearn.datasets import load_diabetes
from sklearn.metrics import mean_squared_error

diabetes = load_diabetes()
X_r, y_r = diabetes.data, diabetes.target
X_tr, X_te, y_tr, y_te = train_test_split(X_r, y_r, test_size=0.2, random_state=42)

pipe_reg = Pipeline([('sc', StandardScaler()), ('knn', KNeighborsRegressor(n_neighbors=7, weights='distance'))])
pipe_reg.fit(X_tr, y_tr)
rmse = mean_squared_error(y_te, pipe_reg.predict(X_te), squared=False)
print(f'RMSE: {rmse:.2f}')

When to Use KNN

Good for small to medium datasets, irregular non-linear boundaries, interpretable baselines, multiclass problems. Avoid for large datasets (O(n) prediction), high-dimensional data (curse of dimensionality), low-latency production. Always scale features before using KNN.

Conclusion

KNN is perfect for learning core ML concepts — distance, decision boundaries, bias-variance tradeoff. Always scale features, cross-validate to choose k. For production scale, use FAISS or Annoy for approximate nearest neighbours.