📋 KEY INSIGHTS
- Vector databases store, index, and query high-dimensional embedding vectors — the dense numerical representations produced by embedding models (BERT, Sentence-BERT, CLIP, OpenAI text-embedding-3) — using approximate nearest-neighbour (ANN) algorithms that are orders of magnitude faster than exact nearest-neighbour search. They are the retrieval backbone of every RAG system, semantic search engine, and embedding-based recommendation system.
- The four main ANN index types have distinct performance profiles: HNSW (Hierarchical Navigable Small World graphs) achieves the best recall-speed tradeoff and is the default index in Qdrant, Weaviate, and Milvus; IVF-PQ (Inverted File with Product Quantisation) compresses vectors 8-32× and is optimal for very large datasets (100M+ vectors) where memory is the binding constraint; Annoy (Spotify) uses random projection trees and is efficient for static datasets with batch queries; ScaNN (Google) uses anisotropic vector quantisation and achieves state-of-the-art ANN performance on billion-scale benchmarks.
- Hybrid search — combining dense vector similarity search with sparse BM25 keyword search — consistently outperforms either method alone on real-world retrieval benchmarks. Users often mix conceptual queries (better handled by dense vectors) with exact keyword queries (better handled by BM25). Reciprocal Rank Fusion (RRF) or learned re-ranking models combine the two ranked lists into a single ranking.
- Filtered vector search — searching only over a subset of vectors that satisfy a metadata filter (e.g. only documents from a specific date range, category, or user) — is the most common production requirement and is where vector databases differ most in capability. HNSW with pre-filtering is fastest but requires building one index per filter combination; post-filtering scans the full index and discards non-matching results (high recall but potentially slow); in-graph filtering (Qdrant’s payload indexing, Weaviate’s where filter) is the correct approach for complex, dynamic filters.
- Embedding freshness — keeping vector indices up-to-date as the underlying document corpus changes — is a significant operational challenge. Append-only index updates (new documents only) are straightforward; deletion and update require segment-based architectures (like Elasticsearch) that rebuild index segments asynchronously. For RAG applications where documents change frequently (internal wikis, live databases), the indexing pipeline must be designed for incremental updates from the start.
- The practical choice between managed vector database services (Pinecone, Weaviate Cloud, Zilliz) and self-hosted options (Qdrant, Milvus, pgvector) is primarily a build-vs-buy decision: managed services reduce operational overhead but add cost at scale; self-hosted options are cheaper at scale but require infrastructure expertise. For teams already using PostgreSQL, pgvector adds ANN capabilities with zero additional infrastructure.
Semantic search — finding documents that are conceptually similar to a query, rather than documents containing the exact query words — has become the default search paradigm for knowledge retrieval in 2026. Behind every RAG system, every AI-powered internal search tool, and every embedding-based recommendation feature is a vector index: a data structure that stores millions of high-dimensional embedding vectors and answers nearest-neighbour queries in milliseconds. Understanding how vector databases work — the indexing algorithms, the query types they support, the operational challenges of keeping indices fresh, and the tradeoffs between different products — is now a practical requirement for any data scientist building AI-powered applications. This guide covers ANN index algorithms (HNSW, IVF-PQ, ScaNN), the major vector database products and their architectural differences, hybrid search, filtered vector search, embedding pipeline design, and the operational patterns for keeping vector indices up-to-date in production.
Approximate Nearest-Neighbour Algorithms — The Foundation
The core operation of a vector database is k-nearest-neighbour (k-NN) search: given a query vector q and a corpus of N vectors, find the k vectors most similar to q (by cosine similarity or Euclidean distance). Exact k-NN requires computing the distance between q and every vector in the corpus — O(Nd) operations for d-dimensional vectors. For N=10M vectors and d=1536 (OpenAI’s text-embedding-3-small dimensionality), exact search requires 15 billion floating-point operations per query — 150ms at 100 GFLOPS, far too slow for interactive applications. Approximate nearest-neighbour (ANN) algorithms trade a small probability of missing the true nearest neighbours for a 100-1000× speedup, achieving 95-99% recall at query latencies of 1-10ms.
HNSW (Hierarchical Navigable Small World graphs, Malkov and Yashunin, 2018) builds a multi-layer proximity graph where each node is a vector, edges connect nearby vectors, and upper layers contain fewer nodes with longer-range connections (like a skip list for graphs). A query traverses the graph from the top layer downward, greedily moving to the nearest neighbour at each layer, refining the search progressively. HNSW achieves state-of-the-art recall-speed tradeoffs and supports insertions and deletions without full index rebuilds. The M parameter (number of edges per node) and efConstruction (beam width during construction) are the key quality-speed knobs: M=16, efConstruction=200 is a good starting point. The graph neural network perspective on HNSW is illuminating: the proximity graph is a geometric graph that supports efficient routing via greedy graph traversal, and the multi-layer structure is the navigable small-world property that makes long-range shortcuts available at the top layers.
IVF-PQ (Inverted File with Product Quantisation) is the memory-efficient complement to HNSW. Product Quantisation (PQ) compresses each vector by dividing it into M subvectors of dimension d/M, quantising each subvector to one of K centroids (typically K=256), and storing only the M centroid indices instead of the full vector — achieving 8-32× compression. The inverted file (IVF) clusters vectors into nlist centroids, and queries are searched only within the nprobe nearest clusters. The combined IVF-PQ index reduces memory from 6GB (10M × 1536 × 4 bytes float32) to ~20MB (10M × 8 bytes for PQ codes) — a 300× reduction — at the cost of lower recall. IVF-PQ is the standard index for billion-scale corpora where HNSW’s memory requirements (which grow linearly with N×M) become prohibitive. The connection to k-means clustering is direct: IVF cluster centroids are computed by k-means on the embedding corpus, and PQ codebooks are computed by k-means on each subvector space.
| Product | Default Index | Filtered Search | Hybrid Search | Deployment | Best For |
|---|---|---|---|---|---|
| Pinecone | Proprietary (HNSW-like) | Metadata filter | Sparse+dense (serverless) | Managed only | Managed, quick start |
| Qdrant | HNSW | Payload indexing (in-graph) | Sparse+dense native | Self-hosted / Cloud | Best open-source HNSW; filtering |
| Weaviate | HNSW | Where filter | BM25 + vector native | Self-hosted / Cloud | Hybrid search; GraphQL API |
| Milvus | HNSW / IVF-PQ / DiskANN | Scalar index | Sparse+dense | Self-hosted / Zilliz | Billion-scale; production Kubernetes |
| pgvector | IVFFlat / HNSW | Full SQL WHERE | Via tsvector | PostgreSQL extension | Existing Postgres; operational simplicity |
| FAISS | IVF-PQ / HNSW / ScaNN | Manual (no built-in) | Manual | Library (Python/C++) | Research; custom systems |
Hybrid Search, Filtered Queries, and RAG Integration
Hybrid search combines dense vector similarity (semantic similarity) with sparse BM25 keyword matching (exact term overlap), addressing the limitations of each method alone. Dense search excels at semantic matching — “car” matches “automobile” — but misses exact keyword queries (“GPT-4o release date”) where the relevant document contains the exact terms. BM25 excels at exact-term matching but misses semantic synonyms. The standard combination is Reciprocal Rank Fusion (RRF): merge the two ranked lists by assigning each document a score of Σ_i 1/(k + rank_i(doc)) across ranking systems i, where k=60 is a smoothing constant. RRF is hyperparameter-free, robust to score scale differences, and consistently outperforms linear score combinations that require careful weight tuning. The connection to our RAG pipeline is direct: hybrid search is the recommended retrieval strategy for production RAG systems, and our prompt engineering guide covers how to structure the retrieval context for the generator.
The indexing pipeline for a production vector search system: (1) Document chunking — split documents into chunks of 256-512 tokens with overlap, balancing context richness per chunk against retrieval precision; (2) Embedding generation — encode each chunk with an embedding model (Sentence-BERT, E5, OpenAI text-embedding-3) using batch inference; (3) Metadata extraction — extract structured fields (document date, author, category, access permissions) that will be used as filters; (4) Index upsert — write vectors and metadata to the vector database; (5) BM25 index update — update the sparse index (Elasticsearch, Qdrant’s sparse vectors, Weaviate’s inverted index) with the same chunks. This pipeline is an ETL pipeline in the modern lakehouse architecture: raw documents land in the Bronze layer, chunked and embedded documents flow through the Silver layer, and the vector index is a Gold-layer artefact consumed by the RAG application. The word embeddings guide covers the embedding models used in this pipeline, and the data quality guide covers freshness and consistency guarantees for the vector index.
✦ SUMMARIZE THIS ARTICLE WITH AI
ChatGPTWord embeddings and the embedding models (BERT, Sentence-BERT, E5) that vector databases store are in our Word Embeddings guide. RAG systems and prompt engineering that consume vector database retrieval are in our Prompt Engineering guide and Generative AI guide. Clustering algorithms (k-means for IVF centroids, HNSW graph structure) are in our Clustering guide. ETL pipeline design for the indexing pipeline is in our ETL Pipelines guide. Modern data architecture — the lakehouse layer where vector indices fit — is in our Modern Data Architecture guide. Knowledge graphs as a structured complement to vector search for relational queries are in our Knowledge Graphs guide.
