More context makes it worse.

// the mechanism

There is one budget and everything shares it. The system prompt, the conversation so far, every retrieved chunk, every tool result, and the answer itself all come out of the same window. They are not stored separately — they compete.

Anthropic frames it as an attention budget: context is a finite resource with diminishing marginal returns. Past a point, adding relevant material stops helping and starts costing you.

Length alone degrades accuracy

This is measured, not folklore. Chroma's context-rot study ran 18 models and found performance degrades as input length grows — a single distractor lowers accuracy, and four compound the effect. RULER shows effective context is far below the advertised number; NoLiMa found 11 of 13 models dropped below half their short-context baseline at 32K.

The middle gets under-weighted

Attention is not uniform across the window. The classic "lost in the middle" result found a U-shaped curve: material at the start and end is used well, material in the middle is used worst. In one setup, moving the key document from best position to the middle cost about 22 points — and the middle placement scored below the same model given no documents at all.

Newer models handle this better. None are immune. If one passage decides the answer, do not bury it at position 40 of 80.

Wrong context actively misleads

Irrelevant context dilutes; contradictory context overrides. Related-but-wrong passages have been measured dropping accuracy by 5.9–11.1 points, and a plausible contradiction can cost around 15.8. Models will adopt a counterfactual passage over their own prior knowledge roughly 65% of the time.

Concretely: a stale line saying an order is "still processing" when it shipped last week doesn't confuse the model — it convinces it.

The rule

That phrasing is close to Anthropic's own: the smallest possible set of high-signal tokens that maximise the likelihood of your desired outcome. Practically — retrieve fewer, better chunks; put what decides the answer at the start or the end; and prune stale material aggressively, because it outranks being merely useless.

This is what context engineering names: the layer that includes prompt design, retrieval, memory and tool output as components. Related: context rot and context window limits.

Sources: Anthropic — effective context engineering · Chroma — context rot · Liu et al. — lost in the middle · NoLiMa · context-faithful prompting