Recommendation Systems Explained – Collaborative Filtering, Matrix Factorisation and Two-Stage Retrieval

Recommendation systems are among the highest-value machine learning applications in production — Netflix attributes over 80% of content watched to its recommendation engine, and Amazon’s “customers also bought” section drives a third of its revenue. Yet despite their impact, recommendation systems require a fundamentally different approach to ML: the data is sparse (most users have rated a tiny fraction of available items), feedback is often implicit rather than explicit, and the evaluation must account for ranking quality rather than simple prediction accuracy. This guide covers the full stack — from the mathematical foundations of collaborative filtering and matrix factorisation to two-stage production architectures and evaluation.

Collaborative Filtering — User-Based and Item-Based

Collaborative filtering (CF) makes recommendations based on the collective behaviour of all users, without needing any information about the items themselves. The intuition: if user A and user B have similar taste (they both liked the same 10 films), then A’s rating on a film B hasn’t seen is a good predictor of B’s future rating. CF comes in two flavours: memory-based (direct similarity computation) and model-based (learn latent factors from the interaction matrix).

User-based CF: Find users similar to the target user (using cosine similarity or Pearson correlation on their rating vectors), then predict ratings as a weighted average of similar users’ ratings. Scales poorly — computing similarity for millions of users is O(n²). Item-based CF: Compute similarity between items (which is more stable over time than user similarity), then recommend items similar to what the user already liked. Amazon’s original patent used item-based CF for scalability.

import numpy as np
import pandas as pd
from sklearn.metrics.pairwise import cosine_similarity

# User-item rating matrix (rows=users, cols=items, 0=not rated)
ratings = pd.DataFrame({
    'item_A': [5, 4, 0, 0, 1],
    'item_B': [3, 0, 4, 0, 1],
    'item_C': [4, 5, 0, 2, 0],
    'item_D': [0, 4, 5, 4, 0],
    'item_E': [0, 0, 3, 5, 4],
}, index=['user_1','user_2','user_3','user_4','user_5'])

# Item-based CF: cosine similarity between items (columns)
item_sim = pd.DataFrame(
    cosine_similarity(ratings.T),
    index=ratings.columns, columns=ratings.columns
)

def recommend_items(user_id, n=3):
    user_ratings = ratings.loc[user_id]
    unrated = user_ratings[user_ratings == 0].index.tolist()
    scores = {}
    for item in unrated:
        # Weighted sum: similar items the user HAS rated
        rated_items = user_ratings[user_ratings > 0].index
        sim_scores  = item_sim[item][rated_items]
        user_scores = user_ratings[rated_items]
        scores[item] = (sim_scores * user_scores).sum() / (sim_scores.abs().sum() + 1e-9)
    return sorted(scores, key=scores.get, reverse=True)[:n]

print('Recommendations for user_1:', recommend_items('user_1'))

Matrix Factorisation — SVD and ALS

Model-based CF learns compact latent representations of users and items. The insight: a user-item rating matrix R (m users × n items) can be approximated as the product of two lower-rank matrices: R ≈ U × V^T, where U is (m × k) user embeddings and V is (n × k) item embeddings, and k is the number of latent factors (typically 20–200). These latent factors implicitly capture concepts like “action fan”, “budget-conscious”, “early adopter” without being explicitly programmed.

from implicit import als     # pip install implicit
import scipy.sparse as sp

# Build sparse user-item matrix (implicit feedback: purchase counts)
# rows = users, cols = items, values = interaction counts
interactions = sp.csr_matrix(ratings.values.astype('float32'))

# ALS model — designed for implicit feedback
model = als.AlternatingLeastSquares(
    factors=50,         # latent dimensions
    regularization=0.1,
    iterations=20,
    alpha=40,           # confidence scaling for implicit data
    use_gpu=False
)
model.fit(interactions)  # note: ALS expects item-user matrix in implicit lib

# Get top-N recommendations for user 0
user_items = interactions[0]   # user 0's interactions
ids, scores = model.recommend(0, user_items, N=3, filter_already_liked_items=True)
print('Top-3 items for user 0:', ids, 'Scores:', scores.round(3))

# Find similar items
similar_ids, sim_scores = model.similar_items(0, N=5)
print('Items similar to item 0:', similar_ids)

Production Architecture — Two-Stage Retrieval and Ranking

Production recommendation systems handling millions of users and items cannot score every user-item pair at request time. The industry standard is a two-stage pipeline. Stage 1 — Candidate Retrieval: Retrieve ~100–1000 plausible candidates from the full catalogue very fast (typically <50ms). Methods include ANN (Approximate Nearest Neighbour) search over learned embeddings (FAISS, ScaNN), or simple rules (popular items, items from followed creators). Stage 2 — Ranking: Score the ~1000 candidates with a richer model (gradient boosting or neural network) that uses user features, item features, context features, and interaction history. Return top-K ranked results.

import faiss, numpy as np

# Two-Tower model produces user and item embeddings
# Assume we have pre-trained embeddings from a model
user_embeddings = np.random.randn(10000, 64).astype('float32')  # 10k users
item_embeddings = np.random.randn(50000, 64).astype('float32')  # 50k items
faiss.normalize_L2(item_embeddings)  # L2 normalise for cosine similarity

# Build FAISS index for fast approximate nearest-neighbour search
dim   = 64
index = faiss.IndexFlatIP(dim)     # Inner Product (= cosine after L2 norm)
index.add(item_embeddings)          # Index all items
print('Index size:', index.ntotal)

def retrieve_candidates(user_id, k=100):
    query = user_embeddings[user_id:user_id+1].copy()
    faiss.normalize_L2(query)
    distances, item_ids = index.search(query, k)
    return item_ids[0], distances[0]   # (100 candidates, similarity scores)

candidates, scores = retrieve_candidates(user_id=42, k=100)
print('Retrieved', len(candidates), 'candidates for user 42')
print('Top-5 item IDs:', candidates[:5], 'Scores:', scores[:5].round(3))

Evaluation Metrics and Common Interview Questions

Q: What is the difference between Precision@K and NDCG@K?
Precision@K measures the fraction of recommended items in the top-K that are relevant (e.g. actually purchased). It treats all relevant items equally. NDCG@K (Normalised Discounted Cumulative Gain) is a ranked metric — it rewards putting more relevant items higher in the list. A relevant item at position 1 scores much more than the same item at position 10. NDCG is the standard metric for systems where rank order matters.

Q: How do you handle the cold-start problem?
New user cold-start: ask for explicit preferences during onboarding, use demographic-based or popularity-based recommendations, or use a content-based fallback until enough interactions accumulate. New item cold-start: use content features (title, description, category) to find similar items via content-based filtering, boost new items in exploration slots.

Q: What is exposure bias in recommendation systems?
Models trained on historical interactions only see items that were previously recommended. Popular items get recommended more, accumulate more interactions, and rank higher — creating a feedback loop that suppresses long-tail items. Counterfactual learning and inverse propensity scoring (IPS) are standard techniques to correct for this.

✦ SUMMARIZE THIS ARTICLE WITH AI

ChatGPT Perplexity Claude Grok

For the model evaluation metrics (NDCG, MAP, ranking metrics) applied to recommendation systems, our Model Evaluation guide covers the full framework. The matrix factorisation and ALS concepts connect directly to the neural network approaches in our Neural Network Architectures guide. Feature engineering for user and item side features is covered in our Feature Engineering guide. Deploying recommendation APIs with latency constraints connects to our MLOps Interview Q&A.