Structured Outputs and JSON Schemas

A language model normally replies with free text. A program cannot safely act on free text, because it does not know where the fields are or whether they are valid. Structured output means the model returns data in a fixed shape, usually JSON, and a JSON Schema is the document that describes that shape. Agents depend on this every time they pick a tool, fill its arguments or hand data to another step.

Overview: how structured output works

An agent is a loop: the model decides, your code acts. The decision must be readable by code. If the model says "I think we should maybe open a bug, medium priority", you have to guess. If it returns {"category": "bug", "priority": 3}, you can read it with json.loads and act.

JSON Schema is a standard, JSON-based vocabulary for describing JSON data: which keys exist, their types, which are required and which values are allowed. The same schema language is used in three places in agent work:

• Tool definitions: each tool declares an input schema so the model knows which arguments to produce.
• Structured-output features: many model providers let you pass a schema so the model is constrained to follow it. The exact parameter names differ by provider and change over time, so check your provider's current documentation.
• Your own validation: after the reply arrives, your code checks it against the schema before using it.

Three levels of reliability

1. Prompt only: you write "reply with JSON". Models often comply but can add prose, markdown fences or wrong types.
2. Provider-enforced schema: the provider restricts generation so output matches a schema (often called structured outputs, JSON mode or tool calling). Much more reliable, but support for individual schema keywords varies.
3. Your validation and retry: whatever the provider does, you still validate. This is the layer you control and must never skip.

Valid JSON is not the same as correct JSON. A schema guarantees shape (a priority is an integer from 1 to 5). It cannot guarantee the model chose the right priority. Also, any text the model read earlier, such as a web page or an email, may contain a prompt injection. Treat structured output as untrusted input to your program, just like data from a web form.

Key concepts and syntax

A schema is itself a JSON object. The general form is:

{
  "type": "object",
  "properties": {
    "field_name": { "type": "string" }
  },
  "required": ["field_name"],
  "additionalProperties": false
}

Two details trip people up. First, a key missing from required may simply be absent, so your code must handle that. Second, in Python True is an instance of int, so a careful validator must reject booleans where an integer is expected.

Examples

Example 1: a schema for a support ticket

This schema describes the data an agent must produce when it triages a customer message.

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["billing", "bug", "feature"],
      "description": "What kind of request this is"
    },
    "priority": { "type": "integer", "minimum": 1, "maximum": 5 },
    "summary": { "type": "string", "description": "One sentence, max 100 characters" }
  },
  "required": ["category", "priority", "summary"],
  "additionalProperties": false
}

The category is limited to three words, so the model cannot invent "urgent". The priority is a bounded integer. All three keys are required, and nothing extra is allowed. A valid reply looks like {"category": "bug", "priority": 2, "summary": "Login crashes on submit"}.

Example 2: validate, then retry with the errors

This program has no network dependency. call_llm() is a stub that replays two scripted replies: the first is wrong and wrapped in chatty prose, the second is correct. To use a real model, replace the body of call_llm() with a call to your provider's SDK that takes the message list and returns the reply text. The validator is a small hand-written one covering the keywords in the table; in production you can use a maintained library such as jsonschema instead.

import json

TYPE_MAP = {
    'string': str,
    'integer': int,
    'number': (int, float),
    'boolean': bool,
    'array': list,
    'object': dict,
    'null': type(None),
}

TICKET_SCHEMA = {
    'type': 'object',
    'properties': {
        'category': {'type': 'string', 'enum': ['billing', 'bug', 'feature']},
        'priority': {'type': 'integer', 'minimum': 1, 'maximum': 5},
        'summary': {'type': 'string'},
    },
    'required': ['category', 'priority', 'summary'],
    'additionalProperties': False,
}


def type_ok(value, expected):
    if expected in ('integer', 'number') and isinstance(value, bool):
        return False
    return isinstance(value, TYPE_MAP[expected])


def validate(value, schema, path='$'):
    errors = []
    expected = schema.get('type')
    if expected and not type_ok(value, expected):
        return [f'{path}: expected {expected}, got {type(value).__name__}']
    allowed = schema.get('enum')
    if allowed is not None and value not in allowed:
        errors.append(f'{path}: {value!r} not in {allowed}')
    if isinstance(value, (int, float)) and not isinstance(value, bool):
        low = schema.get('minimum')
        high = schema.get('maximum')
        if low is not None and value < low:
            errors.append(f'{path}: {value} is below minimum {low}')
        if high is not None and value > high:
            errors.append(f'{path}: {value} is above maximum {high}')
    if isinstance(value, dict):
        for name in schema.get('required', []):
            if name not in value:
                errors.append(f'{path}: missing required property {name!r}')
        props = schema.get('properties', {})
        for name, sub in props.items():
            if name in value:
                errors.extend(validate(value[name], sub, f'{path}.{name}'))
        if schema.get('additionalProperties') is False:
            for name in value:
                if name not in props:
                    errors.append(f'{path}: unexpected property {name!r}')
    if isinstance(value, list) and 'items' in schema:
        for i, item in enumerate(value):
            errors.extend(validate(item, schema['items'], f'{path}[{i}]'))
    return errors


REPLIES = [
    'Sure! Here is the ticket: {"category": "urgent", "priority": "high", "summary": "Login crash"}',
    '{"category": "bug", "priority": 2, "summary": "Login crash"}',
]
calls = {'n': 0}


def call_llm(messages):
    # Stub: replace with a real SDK call that returns the reply text.
    reply = REPLIES[calls['n']]
    calls['n'] += 1
    return reply


def extract_json(text):
    start = text.find('{')
    end = text.rfind('}')
    if start == -1 or end == -1:
        raise ValueError('no JSON object found')
    return json.loads(text[start:end + 1])


def get_structured(prompt, schema, max_attempts=3):
    instruction = prompt + '\nReply with JSON matching this schema:\n' + json.dumps(schema)
    messages = [{'role': 'user', 'content': instruction}]
    for attempt in range(1, max_attempts + 1):
        reply = call_llm(messages)
        try:
            data = extract_json(reply)
        except ValueError as exc:
            errors = [str(exc)]
        else:
            errors = validate(data, schema)
            if not errors:
                print(f'attempt {attempt}: valid')
                return data
        print(f'attempt {attempt}: invalid')
        for message in errors:
            print('  -', message)
        messages.append({'role': 'assistant', 'content': reply})
        feedback = 'Your JSON was invalid:\n' + '\n'.join(errors) + '\nReturn corrected JSON only.'
        messages.append({'role': 'user', 'content': feedback})
    raise RuntimeError('model never produced valid output')


if __name__ == '__main__':
    ticket = get_structured('Triage: the login page crashes when I press submit.', TICKET_SCHEMA)
    print(ticket)

Output:

attempt 1: invalid
  - $.category: 'urgent' not in ['billing', 'bug', 'feature']
  - $.priority: expected integer, got str
attempt 2: valid
{'category': 'bug', 'priority': 2, 'summary': 'Login crash'}

The first reply contains prose, so extract_json slices from the first { to the last }. It parses, but fails validation twice. The loop sends the exact error messages back to the model, which is far more useful than a vague "try again". The second reply passes. The attempt limit prevents an endless, costly loop.

Example 3: a typed MCP tool

The Model Context Protocol (MCP) lets an agent discover tools on a server. Each tool advertises a JSON Schema for its inputs. With the official Python SDK you do not write that schema by hand: FastMCP builds it from your type hints. First install the SDK:

pip install "mcp[cli]"

from typing import Literal

from pydantic import BaseModel, Field
from mcp.server.fastmcp import FastMCP

mcp = FastMCP('ticket-server')


class Ticket(BaseModel):
    id: int
    category: Literal['billing', 'bug', 'feature']
    priority: int = Field(ge=1, le=5)
    summary: str


@mcp.tool()
def create_ticket(category: Literal['billing', 'bug', 'feature'], priority: int, summary: str) -> Ticket:
    """Create a support ticket in the demo store."""
    return Ticket(id=1, category=category, priority=priority, summary=summary)


if __name__ == '__main__':
    mcp.run()

Here Literal[...] becomes an enum in the tool's input schema, and the docstring becomes the tool description the model reads. In recent SDK versions, a Pydantic return type also produces a structured result with an output schema; check the SDK documentation for the version you install. Running the file starts a server that an MCP client connects to, so there is no console output to show. Note that the SDK validates arguments against the generated schema, but you should still enforce business rules inside the function.

Under the hood: one structured step

1. Your code sends the prompt plus the schema (as tool input schema, provider structured-output setting, or text).
2. The model generates tokens. With enforced schemas, generation is restricted so it cannot emit invalid structure; without them, it only tries to follow your instructions.
3. You receive text (or an already-parsed tool call).
4. You parse it with json.loads. Parsing can fail even for good models, for example when output is cut off by a token limit.
5. You validate against the schema. This catches wrong types, missing keys and bad enum values.
6. You apply semantic and security checks the schema cannot express: does this ticket id exist, is this file path inside the allowed folder, is this user allowed to do this?
7. For risky actions (deleting data, sending email, spending money) you show the parsed action to a human and wait for approval.

Common Mistakes

Mistake 1: trusting the reply without parsing or validating

# WRONG
data = json.loads(reply)          # fails if the reply has prose or markdown fences
send_refund(data['amount'])       # KeyError if missing; wrong type if a string

# RIGHT
data = extract_json(reply)
errors = validate(data, REFUND_SCHEMA)
if errors:
    raise ValueError(errors)
if data['amount'] > 100:
    ask_human_to_approve(data)    # risky action: human in the loop
else:
    send_refund(data['amount'])

The wrong version crashes on small formatting changes and acts on any value the model (or an injected instruction) produces. The right version parses defensively, validates, and puts a limit and a human approval step on money.

Mistake 2: a schema that is too loose

If you forget required and additionalProperties: false, an empty object {} or an object with surprise keys is valid. Always list required keys, use enum for closed sets, and bound numbers with minimum and maximum.

Mistake 3: believing valid means true

A reply such as {"file": "../../etc/passwd"} can match {"type": "string"} perfectly. Schemas check shape only. Resolve and check paths, IDs and URLs against an allowlist in code.

Best Practices

• Keep schemas small and flat. Deeply nested schemas confuse models and are not equally supported by every provider.
• Use enum instead of free text wherever the choices are known.
• Write a short description for each non-obvious field; models use them.
• Always validate on your side, even when the provider enforces a schema.
• Retry with the concrete error messages, and cap attempts (two or three is usual).
• Handle truncated output and refusals as separate cases; they may not be JSON at all.
• Treat parsed values as untrusted: check permissions, paths, amounts and recipients in code.
• Require human approval for irreversible or outward-facing actions.
• Log the raw reply and the validation errors; they are your best debugging data.
• Check your provider's docs for which schema keywords are supported, because this changes over time.

Practice Exercises

1. Extend the schema. Add an optional tags property: an array of strings. Update the validator call to confirm that {"tags": ["ui", 3]} reports an error at $.tags[1]. (Hint: the validator already handles items.)
2. Add a length rule. Add support for maxLength on strings to validate(), then use it to reject summaries longer than 100 characters.
3. Approval gate. Write a function that takes a validated action dictionary and, if priority is 5, prints the action and asks input() for a yes or no before continuing.

Summary

• Structured output makes model replies machine-readable; JSON Schema describes the exact shape.
• The same schema language powers tool inputs, provider structured outputs and MCP tools.
• Key keywords: type, properties, required, enum, bounds, items and additionalProperties.
• Always parse defensively, validate, and retry with specific error feedback and an attempt limit.
• Schema-valid does not mean safe or correct: add semantic checks, allowlists and human approval for risky actions.
• With FastMCP, Python type hints generate the tool's JSON Schema automatically.