HNSW at Scale: Why Your RAG System Gets Worse as the Vector Database Grows

HNSW (Hierarchical Navigable Small World) algorithms power retrieval in most vector databases but silently degrade RAG system performance as databases grow larger. Through controlled experiments using 200,000 image embeddings from the LAION dataset, researchers found that HNSW recall quality drops faster than flat search as vector counts increase, with recall declining 10%+ when databases grow from 50k to 200k vectors while latency remains stable.