toolcall() ← all concepts

// concept · RAG

Your follow-up question breaks your search

The first question works beautifully. The second one — "which one is cheaper?" — returns nothing useful, because the thing it refers to was never in the string you searched with.

// retrieval only sees what you send it

In a conversational app the second question is usually not self-contained. "Which one is cheaper?" · "What about the annual plan?" · "Does it do that too?" The subject lives in the previous turn, not in the text you embed.

# what the user sees user: Compare the Pro and Team plans agent: … user: Which one is cheaper? # what your retriever receives "Which one is cheaper?" ← no subject. matches nothing.
In conversational search, follow-up questions depend on prior context. Query rewriting helps rewrite the full user query so the AI model doesn't miss any prior context.

Resolve it before you retrieve

The fix is a step you insert ahead of retrieval: take the conversation so far, resolve what the question is actually about, and embed that.

"Which one is cheaper?" ↓ resolve against history "Is the Pro plan or the Team plan cheaper?" ↓ now embed this

This is standard, widely documented practice, and it belongs to a family of techniques — query expansion, decomposition, paraphrasing, multi-query generation, step-back prompting. The conversational case is specifically the coreference one: put the missing noun back.

The catch: a rewrite can change the question

Rewriting is not free, and more rewriting is not better. Two documented failure modes:

over-expansion too many added terms → irrelevant documents semantic drift the rewrite quietly means something else

And the broader finding is blunter than that. Across eight conversational QA datasets, several advanced techniques failed to yield gains and could degrade performance below the no-RAG baseline — effective conversational RAG depended less on method complexity than on whether the retrieval strategy matched the data.

No numbers here on purpose. A widely-quoted "60% of follow-ups have unresolved coreferences" is vendor marketing with no methodology, and the academic work does not isolate a coreference-only ablation — so there is no honest "+N points" figure to quote either. The qualitative claim demonstrates itself in one frame.

The rule

Resolve the question, don't redecorate it. Fill in the missing subject; do not turn a sentence into a pile of keywords. The goal is a query that means exactly what the user meant, stated in full.

Related: why your RAG retrieves garbage covers the chunking side of the same pipeline, reranking fixes ordering rather than phrasing, and working out which part is broken routes you to the right one of the three.

One concept a week. Free.

The deeper, copy-paste version of each ToolCall short — in your inbox.

// total: 0.00 · spam: void · unsubscribe: one click