Dimensionality Reduction – PCA, t-SNE & UMAP for Data Scientists 2026

High-dimensional data is hard to visualise, slow to train on, and often contains redundant features that hurt model performance. Dimensionality reduction compresses data to fewer, more informative dimensions — without losing the structure that matters. This guide covers PCA for linear reduction, t-SNE and UMAP for visualisation, and autoencoders for deep non-linear representations.

Why Reduce Dimensions?

The curse of dimensionality means that as features increase, data becomes increasingly sparse and distances become meaningless. Models overfit, training slows, and visualisation becomes impossible. Dimensionality reduction addresses all three: fewer dimensions mean faster training, less overfitting, and the ability to plot complex data in 2D to reveal clusters and patterns invisible in the original high-dimensional space.

Principal Component Analysis (PCA)

PCA finds the directions of maximum variance in your data (principal components) and projects data onto them. It is linear, fast, and interpretable. Use it as a preprocessing step before ML models, or to identify which features drive the most variation.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.datasets import load_digits

# Load high-dimensional data: 1797 images, each 8x8 = 64 features
digits = load_digits()
X, y   = digits.data, digits.target   # shape (1797, 64)

# Always scale before PCA
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Determine how many components to keep
pca_full = PCA()
pca_full.fit(X_scaled)

# Explained variance ratio
cumvar = np.cumsum(pca_full.explained_variance_ratio_)
n_90 = np.argmax(cumvar >= 0.90) + 1
n_95 = np.argmax(cumvar >= 0.95) + 1
print(f'Components for 90% variance: {n_90}')
print(f'Components for 95% variance: {n_95}')

plt.plot(cumvar)
plt.axhline(0.90, ls='--', color='red',   label='90%')
plt.axhline(0.95, ls='--', color='orange', label='95%')
plt.xlabel('Number of components')
plt.ylabel('Cumulative explained variance')
plt.legend()
plt.show()

# Reduce to 2D for visualisation
pca2 = PCA(n_components=2)
X_2d = pca2.fit_transform(X_scaled)

plt.figure(figsize=(10, 7))
scatter = plt.scatter(X_2d[:, 0], X_2d[:, 1],
                      c=y, cmap='tab10', alpha=0.7, s=20)
plt.colorbar(scatter, label='Digit')
plt.title('PCA: Digits Dataset (2D)')
plt.xlabel(f'PC1 ({pca2.explained_variance_ratio_[0]:.1%} variance)')
plt.ylabel(f'PC2 ({pca2.explained_variance_ratio_[1]:.1%} variance)')
plt.show()

# PCA for preprocessing (reduce to 30 components)
pca30    = PCA(n_components=30, random_state=42)
X_reduced = pca30.fit_transform(X_scaled)
print(f'Reduced: {X.shape} → {X_reduced.shape}')

t-SNE for Visualisation

t-SNE (t-distributed Stochastic Neighbour Embedding) preserves local structure — nearby points in high dimensions stay nearby in 2D. It reveals clusters dramatically better than PCA. Use it purely for visualisation, not for preprocessing before ML (distances are not meaningful).

from sklearn.manifold import TSNE

# Run PCA first to speed up t-SNE on high-dimensional data
pca50 = PCA(n_components=50, random_state=42)
X_pca = pca50.fit_transform(X_scaled)

tsne = TSNE(n_components=2, perplexity=30, n_iter=1000,
            random_state=42, verbose=1)
X_tsne = tsne.fit_transform(X_pca)

plt.figure(figsize=(10, 7))
scatter = plt.scatter(X_tsne[:, 0], X_tsne[:, 1],
                      c=y, cmap='tab10', alpha=0.8, s=20)
plt.colorbar(scatter, label='Digit')
plt.title('t-SNE: Digits Dataset')
plt.axis('off')
plt.show()

# Perplexity controls neighbourhood size (5-50 typical; try multiple)
# Higher perplexity = considers more neighbours = more global structure

UMAP – Faster and More Faithful

UMAP (Uniform Manifold Approximation and Projection) is faster than t-SNE, preserves more global structure, and supports transform() on new data — making it usable as a preprocessing step.

pip install umap-learn

import umap

reducer = umap.UMAP(n_neighbors=15, min_dist=0.1,
                    n_components=2, random_state=42)
X_umap  = reducer.fit_transform(X_scaled)

plt.figure(figsize=(10, 7))
scatter = plt.scatter(X_umap[:, 0], X_umap[:, 1],
                      c=y, cmap='tab10', alpha=0.8, s=20)
plt.colorbar(scatter, label='Digit')
plt.title('UMAP: Digits Dataset')
plt.axis('off')
plt.show()

# UMAP can transform new data (t-SNE cannot)
X_test_umap = reducer.transform(X_test_scaled)

# 3D UMAP for richer exploration
reducer_3d  = umap.UMAP(n_components=3, random_state=42)
X_3d        = reducer_3d.fit_transform(X_scaled)

Autoencoder for Non-Linear Reduction

import torch
import torch.nn as nn

class Autoencoder(nn.Module):
    def __init__(self, input_dim, latent_dim):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, 128), nn.ReLU(),
            nn.Linear(128, 64),        nn.ReLU(),
            nn.Linear(64, latent_dim)
        )
        self.decoder = nn.Sequential(
            nn.Linear(latent_dim, 64), nn.ReLU(),
            nn.Linear(64, 128),        nn.ReLU(),
            nn.Linear(128, input_dim), nn.Sigmoid()
        )

    def forward(self, x):
        z    = self.encoder(x)
        recon = self.decoder(z)
        return recon, z

X_tensor = torch.FloatTensor(X_scaled)
ae = Autoencoder(input_dim=64, latent_dim=2)
optimizer = torch.optim.Adam(ae.parameters(), lr=1e-3)

for epoch in range(200):
    ae.train()
    optimizer.zero_grad()
    recon, z = ae(X_tensor)
    loss     = nn.MSELoss()(recon, X_tensor)
    loss.backward()
    optimizer.step()
    if epoch % 50 == 0:
        print(f'Epoch {epoch}: Loss = {loss.item():.4f}')

ae.eval()
with torch.no_grad():
    _, X_ae = ae(X_tensor)
    X_ae = X_ae.numpy()

plt.figure(figsize=(10, 7))
plt.scatter(X_ae[:, 0], X_ae[:, 1], c=y, cmap='tab10', alpha=0.7, s=20)
plt.title('Autoencoder Latent Space')
plt.show()

Choosing the Right Method

Use PCA when you need a linear, interpretable reduction — for preprocessing before ML models or for understanding which directions drive variance. Use t-SNE when you want the best 2D visualisation of cluster structure and do not need to transform new data. Use UMAP when you want a fast alternative to t-SNE that also preserves global structure and supports transform() on new data. Use autoencoders when your data has complex non-linear structure that PCA misses, or when you need a deep latent representation for generative modelling.

Conclusion

Dimensionality reduction is both a preprocessing tool (PCA before ML) and an analytical tool (t-SNE/UMAP for exploration). Always scale your data before applying any of these methods. Always use PCA as a fast initial step before t-SNE on high-dimensional data — it dramatically speeds up the algorithm with minimal information loss. And remember: UMAP and t-SNE are for visualisation and exploration, not for drawing definitive conclusions about cluster boundaries, which are sensitive to hyperparameter choices.