Pure vector search feels magical — until it confidently returns something that's semantically close but factually wrong. When I built a retrieval pipeline for medical code lookup, this failure mode showed up constantly.
The problem with vectors alone
Embeddings capture meaning, not exact tokens. Ask for a specific code like
E11.9 and a vector model happily returns codes that are "about diabetes" —
useful context, wrong answer. Exact identifiers, rare terms, and acronyms are
exactly where dense retrieval is weakest.
Keyword search (BM25) has the opposite profile: great at exact matches, blind to paraphrase. Search "high blood sugar" and it won't connect to "hyperglycemia."
Hybrid: use both, then re-rank
The fix is to run both retrievers and fuse the results:
- Dense retrieval for semantic recall.
- Sparse (BM25/keyword) retrieval for exact-term precision.
- Fuse the two ranked lists — Reciprocal Rank Fusion is a simple, strong default.
def rrf(dense_ranks, sparse_ranks, k=60):
scores = {}
for ranking in (dense_ranks, sparse_ranks):
for rank, doc_id in enumerate(ranking):
scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
return sorted(scores, key=scores.get, reverse=True)
RRF needs no score normalization and no tuning to get started — it just rewards documents that rank well in either list.
What I learned
- Start with hybrid, not pure vector. The precision floor is much higher.
- Measure on real queries, not vibes. A handful of adversarial lookups exposes more than any benchmark average.
- Keep a keyword escape hatch for identifiers. Users search for exact codes far more than embedding demos suggest.
More on chunking and evaluation in a future post.