// the problem
Vector search uses a bi-encoder: it turns your query into one vector and each chunk into one vector, then compares them. Fast and indexable — but it has to compress every possible meaning of a chunk into a single vector, so it's lossy. The result is a rough similarity ranking where the genuinely best chunk can land low.
And low is fatal. LLM recall degrades as you stuff the context window — a right-but-low-ranked chunk near the bottom often gets skipped. Same retrieval, wrong order, vague answer.
The fix
A cross-encoder reranker passes the query and a chunk through the transformer together, and outputs one relevance score. Because attention runs across both at once, every query token can attend to every chunk token — it models interactions a single static vector can't. Far more accurate. You run it over your retrieved chunks and reorder them.
So why not rerank everything?
There's no precomputed index. A cross-encoder has to run a full transformer inference for every query+chunk pair, at query time. That doesn't scale to millions of docs — Pinecone notes that scoring 40M records with a small reranker would take more than 50 hours, versus under 100 ms for vector search. So you use it as a precision pass, not a search engine.
The two-stage pattern
Retrieve many candidates cheaply with the bi-encoder (high recall), rerank a few with the cross-encoder (high precision), keep the top of those for the LLM. It's one bolt-on step over the top-K you already pull.
Managed rerankers (Cohere Rerank, Pinecone Rerank) are one API call; open cross-encoders (sentence-transformers ms-marco-MiniLM, BGE rerankers) run self-hosted on your own GPU. Either way it's the same one extra step.
The caveats
Reranking is cheap to add, but it isn't free and it isn't magic:
Sources: sentence-transformers — Retrieve & Re-Rank · Cohere — Rerank best practices · Pinecone — Rerankers & two-stage retrieval · Pinecone — Introducing reranking · Hugging Face — sentence-transformers
One concept a week. Free.
The deeper, copy-paste version of each ToolCall short — in your inbox.
// total: 0.00 · spam: void · unsubscribe: one click
