toolcall() ← all concepts

// concept · agent memory

The model is stateless.

Every call is a blank slate — the model remembers nothing between steps. "Memory" is something you build around it: a scratchpad for now, a store for next time.

// no memory between calls

Each API call is independent. The only thing the model "knows" is what's in the context window right now, and that window is finite. Chat feels like it remembers only because the app re-sends the whole history every turn — that's client-side state, not the model's.

So an agent running a multi-step loop forgets everything between steps unless you carry the state forward yourself.

The scratchpad = working memory

Each loop, the agent writes a thought, takes an action, records what it observed — and that running trace is fed back into the next step. That's the ReAct loop, and the trace is the agent's working memory.

# the scratchpad, appended every loop thought: I need the order id action: lookup(order) observation: shipped Monday thought: answer the user # …all fed back in as context next step

Two kinds of memory

short-term

working memory

The scratchpad. Lives inside the context window. Scoped to this one task; gone when the task ends.

long-term

external store

A file, DB, or vector store the agent writes to and reads back — persists across sessions (semantic, episodic, procedural memory).

Because the window is finite (and gets worse as it fills — "context rot"), you can't just keep appending. The pattern is write notes outside the window, pull the relevant ones back in — Anthropic calls it structured note-taking; MemGPT (now Letta) frames it as paging memory in and out like an operating system.

Without memory, an agent loses the goal on long tasks, lets its plan get truncated out of context, repeats work, and forgets everything the moment the session ends.

Sources: Anthropic — context engineering & note-taking · ReAct (Yao et al.) · LangChain — memory (short vs long-term) · MemGPT / Letta

One concept a week. Free.

The deeper, copy-paste version of each ToolCall short — in your inbox.

// total: 0.00 · spam: void · unsubscribe: one click