toolcall() ← all concepts

// concept · context window

Not memory. A token budget.

It feels like your AI remembers the conversation. It doesn't. The context window is the most tokens the model can read in a single call — and that one number constrains more than people think.

// what it actually is

The context window is the maximum number of tokens the model can attend to in one call — the prompt it reads plus the answer it writes. It's per-request working memory, not long-term recall. How many words fit in a token depends on the model's tokenizer: on Claude Opus 4.7 and later, 1M tokens is roughly 555k words (Anthropic's figure), where earlier Claude models fit about 750k. So the count is the model's, not yours.

# one call = one fixed box [ prompt + replayed history | the answer ] # everything must fit in here, together

Input and output share it

It's one budget, two claimants. A big input leaves less room for the reply — pour enough into the box and the model runs out of space to answer. Push past the edge on a hosted API and the call doesn't truncate quietly; it errors. (A local runtime can differ: on its default path Ollama cuts the prompt from the start with a server-log warning, "truncating input prompt", and no error reaches your app; see Ollama's 4K default.)

small prompt -> lots of room to answer huge prompt -> little room left for the answer prompt + answer > window -> context-window-exceeded

It resets every call

The model is stateless. Between turns it remembers nothing. The "it knows what we talked about" feeling is the app replaying the whole conversation on every request — re-sending past turns as input each time. Stop re-sending them and the memory is gone.

# turn 3 isn't "continued" — it's resent request = [ turn1, turn2, turn3 ] # all of it, again # the model never saw turns 1–2 "before" — only now
Bigger isn't free. You pay per token, so a fuller window costs more and runs slower — and quality can sag well before the hard limit (long-context recall degrades, especially in the middle). That last part is its own story → context rot.

Sources: Anthropic — Context windows · OpenAI — Conversation state (stateless) · OpenAI — What are tokens · Liu et al. — Lost in the Middle

One concept a week. Free.

The deeper, copy-paste version of each ToolCall short — in your inbox.

// total: 0.00 · spam: void · unsubscribe: one click