Clustering Algorithms Compared – K-Means, DBSCAN, Hierarchical Python

Clustering finds hidden structure in unlabelled data by grouping similar points together. But not all clustering algorithms are created equal — K-Means fails on non-spherical clusters, DBSCAN handles arbitrary shapes but needs careful tuning, and hierarchical clustering works without specifying K upfront. This guide compares them all with Python code so you can choose the right one for your data.

K-Means Clustering

K-Means is the most popular clustering algorithm. It partitions data into K clusters by iteratively assigning points to the nearest centroid and updating centroid positions.

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
import numpy as np

# Always scale before clustering
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Elbow method to find optimal K
inertias = []
K_range = range(2, 12)
for k in K_range:
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    km.fit(X_scaled)
    inertias.append(km.inertia_)

# Fit final model
km = KMeans(n_clusters=4, random_state=42, n_init=10)
labels = km.fit_predict(X_scaled)

Strengths: fast, scalable (O(n)), works well on spherical clusters with similar sizes. Weaknesses: requires specifying K; assumes spherical clusters; sensitive to outliers; non-deterministic (use n_init=10).

Silhouette Score – Evaluating Cluster Quality

from sklearn.metrics import silhouette_score, davies_bouldin_score

sil = silhouette_score(X_scaled, labels)
db  = davies_bouldin_score(X_scaled, labels)

print(f"Silhouette: {sil:.3f}")  # Higher is better (-1 to 1)
print(f"Davies-Bouldin: {db:.3f}")  # Lower is better

DBSCAN – Density-Based Clustering

DBSCAN groups points that are densely packed together and labels outliers as noise (-1). Unlike K-Means, it finds arbitrary-shaped clusters and doesn’t need K specified upfront.

from sklearn.cluster import DBSCAN
from sklearn.neighbors import NearestNeighbors

# Find optimal epsilon using k-distance plot
nbrs = NearestNeighbors(n_neighbors=5).fit(X_scaled)
distances, _ = nbrs.kneighbors(X_scaled)
distances = np.sort(distances[:, -1])
# Look for the "elbow" in this plot — that's your epsilon

dbscan = DBSCAN(eps=0.5, min_samples=5)
labels = dbscan.fit_predict(X_scaled)

n_clusters = len(set(labels)) - (1 if -1 in labels else 0)
n_noise    = (labels == -1).sum()
print(f"Clusters found: {n_clusters}, Noise points: {n_noise}")

Strengths: finds arbitrary shapes; handles noise/outliers; no K required. Weaknesses: struggles with varying-density clusters; sensitive to eps and min_samples; doesn’t work well in high dimensions.

Agglomerative Hierarchical Clustering

Hierarchical clustering builds a tree (dendrogram) of clusters by merging the closest pair at each step. It’s interpretable and doesn’t require K upfront — you choose the cut level after seeing the dendrogram.

from sklearn.cluster import AgglomerativeClustering
from scipy.cluster.hierarchy import dendrogram, linkage
import matplotlib.pyplot as plt

# Plot dendrogram to choose number of clusters
Z = linkage(X_scaled[:500], method='ward')  # sample for speed
plt.figure(figsize=(12, 5))
dendrogram(Z, truncate_mode='lastp', p=30)
plt.xlabel("Sample index"); plt.ylabel("Distance")
plt.title("Dendrogram"); plt.show()

# Fit with chosen K
agg = AgglomerativeClustering(n_clusters=4, linkage='ward')
labels = agg.fit_predict(X_scaled)

Linkage options: ward (minimises within-cluster variance — usually best), complete (max distance), average, single (min distance — creates chain-like clusters).

Gaussian Mixture Models (GMM)

GMM is a probabilistic model that assumes data comes from a mixture of Gaussian distributions. Unlike K-Means, it provides soft assignments (probabilities) and handles elliptical clusters.

from sklearn.mixture import GaussianMixture

# BIC to choose number of components
bics = []
for k in range(2, 10):
    gmm = GaussianMixture(n_components=k, random_state=42)
    gmm.fit(X_scaled)
    bics.append(gmm.bic(X_scaled))

# Fit best model
gmm = GaussianMixture(n_components=4, covariance_type='full', random_state=42)
gmm.fit(X_scaled)
labels = gmm.predict(X_scaled)
probs  = gmm.predict_proba(X_scaled)  # soft assignments

Which Clustering Algorithm Should You Use?

K-Means is the right starting point for large datasets with roughly spherical clusters where you have a rough idea of K. Use DBSCAN when your clusters have arbitrary shapes, or when you need automatic outlier detection. Choose hierarchical clustering when interpretability matters or when you don’t want to commit to a K upfront — the dendrogram helps you decide. Use GMM when clusters might be elliptical or when you need probability estimates rather than hard assignments.

Complete Comparison Script

from sklearn.cluster import KMeans, DBSCAN, AgglomerativeClustering
from sklearn.mixture import GaussianMixture
from sklearn.metrics import silhouette_score

algorithms = {
    'K-Means':       KMeans(n_clusters=4, random_state=42, n_init=10),
    'DBSCAN':        DBSCAN(eps=0.5, min_samples=5),
    'Agglomerative': AgglomerativeClustering(n_clusters=4, linkage='ward'),
    'GMM':           GaussianMixture(n_components=4, random_state=42),
}

for name, algo in algorithms.items():
    labels = algo.fit_predict(X_scaled)
    n_clusters = len(set(labels)) - (1 if -1 in labels else 0)
    if n_clusters > 1:
        sil = silhouette_score(X_scaled, labels)
        print(f"{name:15s} | Clusters: {n_clusters} | Silhouette: {sil:.3f}")

Conclusion

There is no universally best clustering algorithm — the right choice depends on your data shape, size, and whether you need hard or soft assignments. Start with K-Means for speed, check DBSCAN if you suspect non-spherical clusters or outliers, and use the silhouette score to compare approaches objectively. Always visualise your clusters in 2D (using PCA or UMAP) to validate that the algorithm found meaningful groups.