The hybrid that was worse than its best half
Merging full-text search with vector search scored below vector search alone, with four different embedding models. A reranker fixed it, and the reason says where each half helps.
Date: 2026-11-09. Context: adding embeddings to retrieval (feat/rag-hybrid).
What happened
The plan said "hybrid retrieval": run a full-text search and a vector search, merge them. Everyone does it, so I built it that way and expected it to be the best of the three.
On 40 questions about the project's own docs, the three searches scored:
| Search | recall@1 | recall@5 | recall@10 | MRR |
|---|---|---|---|---|
| Full-text | 25% | 57% | 65% | 0.39 |
Vector (OpenAI text-embedding-3-small) | 48% | 78% | 85% | 0.62 |
| Hybrid (reciprocal rank fusion) | 35% | 80% | 83% | 0.54 |
The hybrid was worse than the vector search alone, and not by chance: with Qwen3, Voyage and Cohere embeddings the hybrid also lost to its own vector half (0.45 against 0.53, 0.45 against 0.53, 0.48 against 0.57). Putting a weak list beside a good one pulled the good one down.
Three experiments said why:
- Weight the weak list less. A lexical weight of 1, 0.5 and 0.25 gave MRR 0.535, 0.552 and 0.566. Better, never as good as 0.617.
- Different kind of query. I added eight searches typed the way a developer types them:
TS2769,bench:recount,markCovered. There the order flipped: full-text 0.69, vector 0.45, hybrid 0.51. - A reranker on top. Taking the hybrid's best 30 candidates and sorting them with a reranking model gave MRR 0.76 on the questions and 0.72 on the keywords, and recall@10 of 93% and 100%. Better than every other row, on both kinds of query.
What we learned
- Fusion is not free. Reciprocal rank fusion trusts the order of each list. If one list is mostly wrong at the top, giving it an equal vote adds wrong answers to the right ones. It only helps when both lists are about equally good, or when they are good at different things and the merge keeps both.
- The two searches are good at different things. Vectors find a text that says the same in other words ("compile errors" for "type errors"). Words find an exact identifier that a vector blurs. A dataset of only sentences would have shown the hybrid as a pure loss, and one of only identifiers as a pure win.
- The hybrid's job is to find candidates, not to rank them. It gets the answer into the top 30 for both kinds of query (recall@10 83% and 100%). A reranker, which reads the question and each candidate together, is the stage that puts it first.
- Measure each stage. "Hybrid, then rerank" sounds like three improvements stacked. The middle one lowered the score and only the last made the stack worth building. Without the numbers per stage the pipeline would have looked fine and been wrong in one place.
What it means for Vektos
The retrieval for the chat is: lexical and vector candidates, merged, then reranked. Each stage has an eval row, so a change to one shows up in its own number. The same habit applies to the benchmark itself: anecdote 08 was two graders counting one mistake; this is two searches voting on one answer. In both, a combination needs its own measurement.
Follow-ups
- Eight keyword searches are a hint, not a result. A bigger set, with real questions from someone who did not write the docs, would tell if the hybrid's recall is stable.
- Try other ways to combine the lists (weights chosen per query, or a learned weight) against just reranking, which already works.
- The journal repeats what the guides say, so several "misses" returned the right text from the wrong document. A document type in the ranking, or leaving the journal out of retrieval for questions about how things work, would remove that.
The CSS that was not there
The dashboard built clean and rendered with half its design-system styles missing, and writing it against our own rules hit the same walls the agents hit.
A citation that pointed at the neighbour
A chunker tuned on our own guides merged 19 short WCAG criteria into the criterion before them. The evals still passed, because they checked the words and not the citation.