BM25 vs. Dense Retrieval: The Accuracy and Latency Trade-offs in Search Ranking

In April 2026, the Gemini Embedding 2 model posted a score of 67.71 on the BEIR (Benchmarking Information Retrieval) leaderboard. Measured in normalized Discounted Cumulative Gain at rank 10 (nDCG@10), that figure represents the current ceiling for dense vector retrieval—a neural approach that maps human queries into high-dimensional mathematical space. On the same leaderboard, the traditional lexical algorithm BM25 sits far behind at 42.0.

That 25-point gap on the Massive Text Embedding Benchmark (MTEB) suggests a total victory for semantic search over exact keyword matching. Dense retrieval uses transformer encoders to convert text into vectors of 768 to 1,536 dimensions. Because it measures the cosine similarity between these vectors, it understands that a user searching for "physician" will be satisfied by a document about a "doctor," even if the exact string never appears.

BM25, conversely, is a sparse retrieval method built on term frequency-inverse document frequency (TF-IDF). It requires exact lexical overlap. If a user queries "bicycle repair" and the corpus only contains the phrase "fixing a bike," BM25 returns nothing. Yet despite this rigid limitation, BM25 remains the foundational baseline for nearly every production search system in the world.[1]

The persistence of BM25 comes down to the specific ways in which semantic understanding fails. Dense retrieval models are trained on general text, which makes them highly capable of interpreting broad concepts but remarkably poor at identifying exact, rare strings. When a user searches for a specific error code like "XYZ-1234" or a niche technical term like the "Karatsuba multiplication algorithm," dense models often return semantically adjacent but factually incorrect results, such as "XYZ-1235".

"Sparse retrieval's strengths are the mirror of dense retrieval's weaknesses," notes a May 2026 analysis by data engineering firm Atlan. "Exact keyword matching works precisely for statute names, error codes, product identifiers, and technical jargon."

Then there is the computational cost. BM25 relies on inverted indexes, a data structure that allows it to search billions of documents in single-digit milliseconds. The operational cost runs between $0.01 and $0.10 per one million queries. Dense retrieval, by contrast, requires generating an embedding for every new document—a process that takes 50 to 200 milliseconds per document—and demands memory-intensive approximate nearest neighbor (ANN) indexes like HNSW or FAISS.

These trade-offs compound as the search space grows. A July 2026 controlled study tested retrieval methods across 28 nested corpus tiers, scaling the data volume by a factor of 450. The researchers found that while agentic and dense retrieval systems dominate small datasets, their accuracy erodes as the haystack expands.

Once the corpus crossed the 10-million token threshold, BM25 overtook the neural models. At the largest scale tested, the traditional lexical algorithm beat dense retrieval and autonomous agents by a margin approaching 20 points. As the volume of text increases, the semantic overlap between unrelated documents rises, making exact-term anchoring necessary to filter out noise.

Because the two methods fail on opposite ends of the query spectrum, production systems increasingly refuse to choose between them. The current industry standard is hybrid retrieval, which runs both pipelines in parallel and merges the results using Reciprocal Rank Fusion (RRF).

RRF calculates a combined score using a simple mathematical formula: the inverse of a constant (typically 60) plus the document's rank in each respective system. If a document ranks first under BM25 and tenth under dense retrieval, its combined score is 1/61 plus 1/70, yielding 0.031. This fusion ensures that documents scoring highly in both exact matching and semantic relevance rise to the top.

The empirical results of this dual approach are definitive. On the MS MARCO recall tasks, weighted fusion of dense and sparse retrieval improves recall by up to 580 percent compared to single-method approaches. To reach a recall@1,000 threshold of 0.98, both pipelines are strictly required; neither BM25 nor dense retrieval can achieve that coverage alone.