Few-Shot Examples and Structured Output

Few-shot examples and structured output solve a practical prompt-design problem: prose is easy for people to read, but hard for software to consume reliably. A model may answer correctly yet wrap the answer in a sentence, reorder fields, omit a rare field, or invent a label. Few-shot examples show input-output pairs so the model can infer the pattern. Structured output defines the exact shape the application expects, then validates the response before later code uses it.

In a generative AI engineering workflow, this lesson sits between prompt wording and application integration. You are designing an interface between probabilistic generation and deterministic code. The outcome is a prompt pattern that encourages the intended shape, plus a validation path that rejects, repairs, or retries invalid output.

What Few-Shot Examples Teach

A few-shot prompt contains demonstrations: compact inputs paired with ideal outputs. The model uses them as in-context learning signals, not permanent training data. During generation, it attends to repeated cues such as field names, label choices, tone, ordering, and edge-case handling. If examples classify tickets into billing, technical, or account, the model is more likely to use those labels than synonyms such as payment or login.

Good demonstrations are deliberately chosen to cover the decision boundary. A classification prompt should include common cases, ambiguous cases, and one example showing what to do when information is missing. An extraction prompt should include ordinary text, missing fields, unusual ordering, and values that look similar but mean different things. A transformation prompt should include the formatting rules that matter, because models imitate concrete patterns more reliably than abstract instructions alone.

Few-shot examples also carry costs. Every example consumes context tokens, and similar examples can crowd out useful evidence. Too many examples may make the model copy superficial details rather than focus on the new input. The best set is small, representative, and intentionally diverse.

How Structured Output Works

Structured output means the response must conform to a machine-readable shape, commonly JSON. The prompt describes the shape, the model generates matching text, and application code parses and validates it. A schema turns summarize the ticket into a concrete contract: required keys, types, enumerated labels, nullable fields, nested objects, arrays, and maximum lengths.

The internal flow has four parts. First, the application builds the prompt with instructions, schema, demonstrations, and the new input. Second, the model predicts tokens, influenced by demonstrations and field names. Third, the application parses the response as JSON or another agreed format. Fourth, validators check syntax and semantics. Syntax validation asks, Is this valid JSON? Semantic validation asks, Does this object contain the required fields, allowed labels, and business rules?

This distinction matters. The string {"priority":"urgent"} can be valid JSON but invalid for an application that only allows low, medium, and high. A useful answer surrounded by prose still fails if the parser expects only JSON. Treat parsing and validation failures as normal outcomes, not rare exceptions.

Prompt Anatomy

A robust few-shot structured-output prompt usually contains the task, schema, formatting constraints, demonstrations, and current input. Put the schema near the response location. Use the same field order and value conventions in every demonstration. Clearly separate the current input from examples so the model does not treat it as another demonstration.

def build_ticket_prompt(ticket_text):
    labels = "billing, technical, account"
    return f"""Classify the support ticket.
Return only JSON with keys: category, confidence, reason.
category must be one of: {labels}.
confidence must be a number from 0 to 1.

Example input: I was charged twice for my subscription.
Example output: {{"category":"billing","confidence":0.94,"reason":"The user reports a duplicate charge."}}

Example input: My password reset email never arrives.
Example output: {{"category":"account","confidence":0.86,"reason":"The user cannot complete account access recovery."}}

Ticket input: {ticket_text}
JSON output:"""

print(build_ticket_prompt("The app crashes whenever I upload a CSV file."))

This example does not call a model; it shows deterministic prompt construction. The prompt defines allowed labels, demonstrates the JSON shape, and places the new ticket after the examples. The expected category is technical, because the symptom is an application crash during file upload. The validator still checks the actual response.

Example 1: Simple Classification

The first progressive pattern is label selection. The main design choice is whether the model may create labels or must choose from a closed list. For routing, analytics, or automation, a closed list is usually better. The examples should include each common label and one boundary case where the reason explains the decision.

import json

ALLOWED_CATEGORIES = {"billing", "technical", "account"}

def validate_ticket_result(raw_text):
    data = json.loads(raw_text)
    if set(data) != {"category", "confidence", "reason"}:
        raise ValueError("unexpected fields")
    if data["category"] not in ALLOWED_CATEGORIES:
        raise ValueError("unknown category")
    if not isinstance(data["confidence"], (int, float)):
        raise ValueError("confidence must be numeric")
    if not 0 <= data["confidence"] <= 1:
        raise ValueError("confidence out of range")
    if not isinstance(data["reason"], str) or not data["reason"].strip():
        raise ValueError("reason is required")
    return data

raw = '{"category":"technical","confidence":0.91,"reason":"The ticket reports a crash during file upload."}'
print(validate_ticket_result(raw)["category"])

Output is technical. The important mechanism is the validation boundary. If the model returns {"category":"bug"}, parsing succeeds but semantic validation fails, preventing an unrecognized label from entering routing code.

Example 2: Extraction With Missing Values

Extraction tasks are harder because source text may omit fields. The schema should say how to represent missing information. Avoid making the model guess absent values. Use null when a required field is not present, and reserve empty strings for fields that are present but blank.

import json

REQUIRED_INVOICE_FIELDS = {
    "invoice_id": str,
    "vendor": str,
    "amount_due": (int, float),
    "due_date": (str, type(None)),
}

def validate_invoice(raw_text):
    data = json.loads(raw_text)
    for field, expected_type in REQUIRED_INVOICE_FIELDS.items():
        if field not in data:
            raise ValueError(f"missing field: {field}")
        if not isinstance(data[field], expected_type):
            raise ValueError(f"wrong type for: {field}")
    if data["amount_due"] < 0:
        raise ValueError("amount_due cannot be negative")
    return data

raw = '{"invoice_id":"INV-2044","vendor":"Northwind Lab","amount_due":1820.5,"due_date":null}'
invoice = validate_invoice(raw)
print(invoice["due_date"] is None)

Output is True. Missing due date is represented as JSON null, which becomes Python None. A good few-shot prompt should include one invoice with a due date and one without, otherwise the model may invent a plausible date to complete the pattern.

Example 3: Nested Output and Repair

The third pattern is nested output, where validation checks arrays and objects inside the top-level result. Nested structures are useful for plans, rubric grading, entity extraction, and tool arguments, but they increase failure paths. A practical system may perform one constrained repair attempt: feed the validation error and invalid JSON back to the model, asking for only corrected JSON. The repair prompt should fix structure, not add facts.

import json

ALLOWED_ACTIONS = {"email", "refund", "escalate"}

def validate_action_plan(raw_text):
    data = json.loads(raw_text)
    if not isinstance(data.get("actions"), list):
        raise ValueError("actions must be a list")
    for index, action in enumerate(data["actions"]):
        if action.get("type") not in ALLOWED_ACTIONS:
            raise ValueError(f"invalid action type at {index}")
        if not isinstance(action.get("rationale"), str) or not action["rationale"].strip():
            raise ValueError(f"missing rationale at {index}")
    return data

def build_repair_prompt(invalid_json, error):
    return f"""Correct the JSON so it matches the schema.
Do not add new facts. Return only JSON.
Validation error: {error}
Invalid JSON: {invalid_json}"""

try:
    validate_action_plan('{"actions":[{"type":"call","rationale":"Customer requested contact."}]}')
except ValueError as error:
    print(build_repair_prompt("{...}", str(error)).splitlines()[2])

The deterministic output is Validation error: invalid action type at 0. The model used an unsupported action type, call. The correction could map it to an allowed value only if the original task permits that mapping; otherwise the correct behavior is to fail and ask for human review. Repair is for structure, not for laundering uncertain content into confidence.

Design Choices and Trade-Offs

The first design choice is open text versus structured data. Open text is better for exploratory writing and human-only review. Structured data is better when software must route, store, compare, trigger tools, or calculate metrics. Many applications use both: structured fields for machines and a short explanation for users.

The second choice is zero-shot versus few-shot prompting. Zero-shot prompts are shorter and easier to maintain. Few-shot prompts help when labels are domain-specific, outputs have a strict style, or edge-case conventions matter. Stale, biased, or narrow examples can reduce quality. Treat examples as testable assets. When a failure occurs, add a regression case and consider a better boundary example.

The third choice is strict rejection versus retry or repair. Immediate rejection is simplest and safest for high-impact actions. Retry can recover from malformed JSON or a missing required field, but it adds latency and cost. Repair can improve user experience, but it must be bounded. A common pattern is one generation, one repair attempt using the validation error, then a clear failure if validation still fails.

The fourth choice is schema complexity. Deep schemas can represent rich workflows, but every nested field increases mismatch risk. Prefer the smallest structure downstream code needs. Use enums for branching values, nullable fields for absent information, and arrays only when order or multiplicity matters.

Failure Modes and Troubleshooting

Symptom: the parser raises a JSON syntax error. Cause: the model returned prose, Markdown fences, comments, or trailing explanation. Diagnostic steps: log the raw response in a secure development environment, inspect the first and last characters, and compare the prompt’s final instruction with the examples. Correction: ask for only JSON, remove examples that show prose, and retry with the parse error.

Symptom: validation fails because labels vary, such as payment instead of billing. Cause: the enum values are not salient enough, or demonstrations use inconsistent labels. Diagnostic steps: count invalid labels across a test set and identify which inputs produce them. Correction: place allowed labels near the response instruction, use each label in examples, and reject unknown values instead of silently mapping them.

Symptom: the model invents missing fields, such as due dates or account IDs. Cause: examples all contain complete records, so the model learns that every field should be filled. Diagnostic steps: run cases with intentionally missing data and compare outputs with the source text. Correction: include demonstrations where missing values are represented as null, and add validation rules that reject values unsupported by the input when provenance is required.

Symptom: quality improves on hand-picked examples but fails on real traffic. Cause: the few-shot set and evaluation set are too similar or miss production variety. Diagnostic steps: sample anonymized inputs, cluster by task type, and measure failures by cluster. Correction: diversify demonstrations, expand boundary tests, and keep a held-out evaluation set out of the prompt.

Security, Performance, and Reliability

Structured output reduces parsing ambiguity, but it does not make model output trusted. Treat every generated object as untrusted until it passes validation and authorization. If a model produces tool arguments, validate operations, resource identifiers, and user permissions before execution. Do not allow {"action":"delete_all"} to drive irreversible behavior merely because it is well-formed JSON.

Performance depends on context size and retries. Few-shot examples increase input tokens, and retries increase latency. Keep examples compact and measure whether each one improves the target metric. Reliability depends on stable schemas, explicit defaults, bounded retries, and clear failure messages. Logging should capture schema version, validation error category, latency, and retry count without storing secrets or unnecessary personal data.

Hands-On Lab: Build a Validated Extractor

Prerequisites: Python 3, a terminal, and basic familiarity with JSON. No model API is required; the goal is to test the deterministic boundary around a model response.

  1. Create structured_output_lab.py.
  2. Add the code below. It builds a few-shot prompt, simulates two responses, validates them, and reports success or failure.
  3. Run python structured_output_lab.py.
  4. Verify that the first response passes and the second fails because its priority is outside the enum.
  5. Cleanup is deleting the lab file. If you later connect this to a real model call, keep the validator and replace only the simulated response function.
import json

ALLOWED_PRIORITIES = {"low", "medium", "high"}

def build_prompt(message):
    return f"""Extract a task record from the message.
Return only JSON with keys: title, owner, priority.
priority must be one of: low, medium, high.
Use null when owner is not named.

Example input: Maya should review the launch checklist today.
Example output: {{"title":"review the launch checklist","owner":"Maya","priority":"medium"}}

Example input: Fix the payment outage immediately.
Example output: {{"title":"fix the payment outage","owner":null,"priority":"high"}}

Message: {message}
JSON output:"""

def validate_task(raw_text):
    data = json.loads(raw_text)
    if set(data) != {"title", "owner", "priority"}:
        raise ValueError("result must contain exactly title, owner, and priority")
    if not isinstance(data["title"], str) or not data["title"].strip():
        raise ValueError("title is required")
    if data["owner"] is not None and not isinstance(data["owner"], str):
        raise ValueError("owner must be a string or null")
    if data["priority"] not in ALLOWED_PRIORITIES:
        raise ValueError("priority is outside the allowed enum")
    return data

def simulated_model_response(case):
    if case == "valid":
        return '{"title":"prepare renewal quote","owner":"Iris","priority":"medium"}'
    return '{"title":"prepare renewal quote","owner":"Iris","priority":"urgent"}'

for case in ["valid", "invalid"]:
    try:
        task = validate_task(simulated_model_response(case))
        print(f"{case}: ok -> {task['priority']}")
    except ValueError as error:
        print(f"{case}: failed -> {error}")

The expected output is deterministic: valid: ok -> medium and invalid: failed -> priority is outside the allowed enum. Verification proves downstream code will not receive an unsupported priority. To extend the lab, add malformed JSON and a missing owner field, then confirm both are rejected for different reasons.

Assessment Exercises

  • You are extracting medication names from clinical notes. Which examples would you include to teach the difference between a current medication, an allergy, and a historical medication? Explain how the schema should represent uncertainty.
  • A model returns valid JSON with an enum value your code does not recognize. Should the application map it to the nearest allowed value, retry, or fail? Defend your answer for a low-risk dashboard and for a payment workflow.
  • Rewrite a prompt that says return a short summary into a structured-output prompt with a schema, two demonstrations, and one missing-data convention.
  • Your validation failure rate rises after adding five examples. List two plausible causes and the diagnostic data you would inspect before changing the model.
  • Design a one-retry repair prompt that corrects schema errors without allowing the model to invent facts.

Summary

Few-shot examples teach the local pattern during the current prompt. Structured output turns the desired response into a contract that software can parse and validate. The combination works best when demonstrations are representative, schemas are small and explicit, missing values have a clear convention, and validators enforce syntax and semantics. Let examples guide generation, but let validation decide whether the result is usable.