toolcall() ← all concepts

// concept · structured outputs

Stop parsing and praying.

Asking the model for JSON and hoping it parses is a coin flip. Bind a schema and malformed output becomes literally impossible — here's why, with the code.

// the problem: prompt-and-pray

You ask for JSON in the prompt and hope. So you get a markdown fence, a chatty intro, a missing key, a hallucinated enum — and JSON.parse throws in production.

# a "prompt-and-pray" response Sure! Here is the JSON: ```json { "name": "Ada", email: ada } # ← missing quotes, missing key, fence, prose ↳ SyntaxError: Unexpected token

The fix: bind a schema

Hand the model a schema and the output is forced to match it.

# OpenAI — Structured Outputs response_format={ "type": "json_schema", "json_schema": { "name": "user", "strict": true, "schema": { "type":"object", "properties": { "name":{"type":"string"}, "email":{"type":"string"} }, "required": ["name","email"], "additionalProperties": false } } } # Claude — strict tool use (or output_config.format, beta) tools=[{ "name":"user", "input_schema": {...}, "strict": true }], tool_choice={ "type":"tool", "name":"user" }

Both ensure valid JSON; only the schema path guarantees your schema (JSON mode alone doesn't).

Why it's guaranteed

It's not the model "trying harder." Structured outputs use constrained decoding: the schema is compiled into a grammar, and at every step the decoder masks any token that would break it. Off-schema output is unsamplable.

# next token must fit the schema — invalid ones are masked "email" ✓ Sure ✗ ``` ✗ banana ✗
OpenAI's own eval: schema compliance went from under 40% (older gpt-4-0613, prompting) to 100% (gpt-4o-2024-08-06 with Structured Outputs). Same family of techniques powers open libraries (Outlines, llama.cpp GBNF grammars).

The one catch

Structure is not correctness.

Shape guaranteed, truth not. A required email field will always be present and a string — but it might be "not-an-email". Constrained decoding fixes the format, never the facts. Still validate values, and know a refusal or a token cap can cut even a valid shape short.

Rule of thumb: the moment another program reads the model's output — extraction, classification, tool arguments, pipelines — give it a schema. Don't parse free text.

Sources: OpenAI — Introducing Structured Outputs · Claude — Structured outputs · Efficient Guided Generation (Outlines) · llama.cpp GBNF grammars

One concept a week. Free.

The deeper, copy-paste version of each ToolCall short — in your inbox.

// total: 0.00 · spam: void · unsubscribe: one click