toolcall() ← all concepts

// concept · context window

Not memory. A token budget.

It feels like your AI remembers the conversation. It doesn't. The context window is the most tokens the model can read in a single call — and that one number constrains more than people think.

// what it actually is

The context window is the maximum number of tokens the model can attend to in one call — the prompt it reads plus the answer it writes. It's per-request working memory, not long-term recall. A token is roughly three-quarters of a word, so the count is the model's, not yours.

# one call = one fixed box [ prompt + replayed history | the answer ] # everything must fit in here, together

Input and output share it

It's one budget, two claimants. A big input leaves less room for the reply — pour enough into the box and the model runs out of space to answer. Push past the edge and the call doesn't truncate quietly; it errors.

small prompt -> lots of room to answer huge prompt -> little room left for the answer prompt + answer > window -> context-window-exceeded

It resets every call

The model is stateless. Between turns it remembers nothing. The "it knows what we talked about" feeling is the app replaying the whole conversation on every request — re-sending past turns as input each time. Stop re-sending them and the memory is gone.

# turn 3 isn't "continued" — it's resent request = [ turn1, turn2, turn3 ] # all of it, again # the model never saw turns 1–2 "before" — only now
Bigger isn't free. You pay per token, so a fuller window costs more and runs slower — and quality can sag well before the hard limit (long-context recall degrades, especially in the middle). That last part is its own story → context rot.

Sources: Anthropic — Context windows · OpenAI — Conversation state (stateless) · OpenAI — What are tokens · Liu et al. — Lost in the Middle

One concept a week. Free.

The deeper, copy-paste version of each ToolCall short — in your inbox.

// total: 0.00 · spam: void · unsubscribe: one click