Tool Calling

Tool calling lets a model ask your application to run a named function instead of inventing facts or pretending it performed an action. The outcome is a controlled loop: the model proposes a tool name and JSON arguments, your application validates and executes the request, then the model uses the result to answer the user.

In the Tools and Agents section, tool calling is the bridge between language reasoning and external capability. It is how a support assistant checks an order, how a coding agent reads a file, and how a data assistant runs a narrow query. Authority stays in your program: the model can request a capability, but code owns schemas, authorization, execution, errors, and result size.

Purpose and Mental Model

A tool is a typed operation exposed to the model with a name, description, and argument schema. The model receives those definitions in the request. When answering requires external data or action, it emits a structured tool call instead of prose. Most APIs represent that call as an assistant message containing an identifier, tool name, and serialized arguments. Your application runs the matching function and sends back a tool-result message correlated to the call identifier.

Think of the model as a planner and argument drafter, not the executor. It may choose the wrong tool, omit a required field, choose a dangerous value, or call a tool when the user has not authorized the action. A good tool has one purpose, a schema that rejects ambiguity, deterministic implementation code, and a compact result for the next model turn.

Internal Mechanism

The loop has five parts. First, the application constructs a model request containing user context and tool definitions. Second, the model decides whether to answer directly or emit one or more tool calls. Third, the application parses each call and validates it against the schema and business rules. Fourth, application code runs the tool with timeouts, permission checks, and typed return values. Fifth, the application sends tool results back so the model can produce a final response or request another tool.

The schema is usually JSON Schema or a related object description. It defines properties, required fields, enumerations, string formats, numeric ranges, arrays, and whether additional properties are accepted. The description helps the model select tools and draft arguments, but descriptions are guidance, not enforcement. Enforcement happens in your parser and validator.

Tool results are also prompt material. If a lookup returns a thousand rows, irrelevant rows compete for context. Results should be bounded, normalized, and shaped for the task: identifiers, statuses, timestamps, totals, and short explanations are usually better than raw database records.

API Anatomy

A practical tool definition contains a stable name, a description written for the model, an input schema, implementation code, and a result contract. Names should be verb phrases such as get_order_status or search_products. Avoid one overloaded execute tool; the model performs better when capabilities are distinct.

WEATHER_TOOL = {
    "name": "get_weather",
    "description": "Return the current weather summary for one city.",
    "parameters": {
        "type": "object",
        "properties": {
            "city": {"type": "string"},
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
        },
        "required": ["city"],
        "additionalProperties": False,
    },
}

print(WEATHER_TOOL["name"])
print(WEATHER_TOOL["parameters"]["properties"]["unit"]["enum"])

This first example defines one read-only tool. The schema says city is required, unit is optional but constrained, and extra keys are rejected. A deterministic run prints the tool name and allowed units. In a real request, this definition would be sent with the user message so the model can emit a call such as get_weather({"city":"Chicago","unit":"fahrenheit"}).

Example: Validate and Dispatch

Dispatch maps the requested tool name to implementation code, validates arguments, applies defaults, and returns a small result. The example uses standard Python so the boundary is visible.

from typing import Any

ALLOWED_UNITS = {"celsius", "fahrenheit"}
WEATHER_DATA = {
    "chicago": {"celsius": 21, "summary": "clear"},
    "seattle": {"celsius": 16, "summary": "light rain"},
}

def c_to_f(value: int) -> int:
    return round(value * 9 / 5 + 32)

def get_weather(city: str, unit: str = "celsius") -> dict[str, Any]:
    key = city.strip().lower()
    if key not in WEATHER_DATA:
        raise ValueError(f"unknown city: {city}")
    if unit not in ALLOWED_UNITS:
        raise ValueError(f"unsupported unit: {unit}")
    row = WEATHER_DATA[key]
    temperature = row["celsius"] if unit == "celsius" else c_to_f(row["celsius"])
    return {"city": key.title(), "temperature": temperature, "unit": unit, "summary": row["summary"]}

def dispatch_tool(name: str, arguments: dict[str, Any]) -> dict[str, Any]:
    if name != "get_weather":
        raise ValueError(f"unknown tool: {name}")
    allowed = {"city", "unit"}
    extra = sorted(set(arguments) - allowed)
    if extra:
        raise ValueError(f"unexpected arguments: {extra}")
    if "city" not in arguments:
        raise ValueError("city is required")
    return get_weather(arguments["city"], arguments.get("unit", "celsius"))

print(dispatch_tool("get_weather", {"city": "Chicago", "unit": "fahrenheit"}))

The expected output is a dictionary for Chicago with temperature 70, unit fahrenheit, and summary clear. Validation is intentionally duplicated with the schema’s intent because the schema helps the model, while the dispatcher protects the system. If the model sends {"city":"Chicago","admin":true}, the dispatcher rejects the extra argument before tool code runs.

Example: A Multi-Turn Tool Loop

A tool call rarely stands alone. The application usually keeps a message list, appends the assistant’s tool call, appends the tool result, and calls the model again. This simulation shows the control flow without a provider API.

from typing import Any

ALLOWED_UNITS = {"celsius", "fahrenheit"}
WEATHER_DATA = {
    "chicago": {"celsius": 21, "summary": "clear"},
    "seattle": {"celsius": 16, "summary": "light rain"},
}

def get_weather(city: str, unit: str = "celsius") -> dict[str, Any]:
    key = city.strip().lower()
    if key not in WEATHER_DATA:
        raise ValueError(f"unknown city: {city}")
    if unit not in ALLOWED_UNITS:
        raise ValueError(f"unsupported unit: {unit}")
    row = WEATHER_DATA[key]
    return {"city": key.title(), "temperature": row["celsius"], "unit": unit, "summary": row["summary"]}

def dispatch_tool(name: str, arguments: dict[str, Any]) -> dict[str, Any]:
    if name != "get_weather":
        raise ValueError(f"unknown tool: {name}")
    allowed = {"city", "unit"}
    extra = sorted(set(arguments) - allowed)
    if extra:
        raise ValueError(f"unexpected arguments: {extra}")
    if "city" not in arguments:
        raise ValueError("city is required")
    return get_weather(arguments["city"], arguments.get("unit", "celsius"))

def fake_model(messages: list[dict[str, object]]) -> dict[str, object]:
    if not any(message.get("role") == "tool" for message in messages):
        return {"role": "assistant", "tool_calls": [{"id": "call_1", "name": "get_weather", "arguments": {"city": "Seattle"}}]}
    tool_result = next(message for message in messages if message.get("role") == "tool")
    data = tool_result["content"]
    return {"role": "assistant", "content": f"Seattle is {data['temperature']} celsius with {data['summary']}."}

messages: list[dict[str, object]] = [{"role": "user", "content": "What is the weather in Seattle?"}]
assistant_message = fake_model(messages)
messages.append(assistant_message)
for call in assistant_message.get("tool_calls", []):
    result = dispatch_tool(call["name"], call["arguments"])
    messages.append({"role": "tool", "tool_call_id": call["id"], "content": result})
final_message = fake_model(messages)
print(final_message["content"])

The expected output is Seattle is 16 celsius with light rain. The final answer is generated only after the tool result is present. Real APIs add parallel tool calls, streaming, call identifiers, and provider-specific fields, but the architecture remains: model request, tool call, validated execution, tool result, final response.

Example: Confirm Before a Write

Read tools and write tools should not share the same risk posture. A model may draft an email, schedule a meeting, or submit a refund, but many applications require explicit confirmation before an irreversible action.

def prepare_refund(user_role: str, order_id: str, amount_cents: int) -> dict[str, object]:
    if user_role != "support_manager":
        return {"status": "denied", "reason": "manager role required"}
    if amount_cents <= 0 or amount_cents > 5000:
        return {"status": "denied", "reason": "amount outside policy"}
    return {"status": "needs_confirmation", "order_id": order_id, "amount_cents": amount_cents}

print(prepare_refund("agent", "A100", 1200))
print(prepare_refund("support_manager", "A100", 1200))

The first call returns denied because the user lacks the required role. The second returns needs_confirmation, not refunded. The model can help prepare an action, while the application decides whether the current identity may commit it.

Design Choices and Trade-Offs

Small tools are easier for the model to choose and easier to authorize. The trade-off is that multi-step tasks may require several model turns. Large tools reduce orchestration overhead but hide important differences, such as read versus write, quote versus purchase, or search versus delete. Prefer separate tools when permissions, side effects, latency, or result shapes differ.

Strict schemas reduce malformed calls and simplify validation. They can require clarification turns when the user provides partial information. Optional fields are useful, but every optional field needs a documented default. Enumerations are better than free text for modes such as units, sort order, or account type because they limit model error and prompt-injection opportunities.

Parallel tool calls reduce latency for independent reads, such as weather plus calendar availability. They are risky for dependent operations because the model may request actions in an order that violates business rules. Use sequential loops when a later decision depends on an earlier result, and make write tools idempotent with a request identifier or confirmation token.

Failure Modes and Troubleshooting

Symptom: the model answers from memory instead of calling a tool. Causes include a vague tool description, missing tool definition, or instructions that allow guessing. Diagnose by logging whether the assistant message contained tool calls and testing prompts that require fresh data. Correct it by making the description task-specific and requiring the tool when the answer depends on external state.

Symptom: invalid arguments, such as missing city or unsupported unit. The cause may be an underspecified schema, ambiguous user request, or model mistake. Diagnose by recording raw tool-call arguments after removing sensitive data. Correct it with stricter schemas, clearer descriptions, and a recovery path that asks the user for the missing field.

Symptom: a successful tool call followed by a wrong final answer. This often happens when the result is too large, inconsistently formatted, or missing the field the model needs. Diagnose by inspecting the exact tool-result message. Correct it by returning a compact object with stable field names and tests that assert the final response reflects tool output.

Symptom: a write action occurs without approval. Causes include trusting model arguments for identity, merging confirmation and execution into one tool, or using broad service credentials. Diagnose with audit logs connecting user identity, model request, tool call, authorization decision, and action result. Correct it by deriving identity from the application session and separating preparation from commit.

Security, Performance, and Reliability

Tool descriptions and results are prompt material, so hostile text from a web page, ticket, or document can try to instruct the model to misuse tools. Treat retrieved text as data, not policy. Do not put secrets in tool results. Do not let the model choose credentials, account identifiers outside the user’s scope, file paths without normalization, or raw SQL.

Performance depends on model turns and tool latency. A single request may become request, tool execution, request, final response. Cache safe read-only results, set timeouts, cap result sizes, and avoid tools that return unbounded data. Reliability improves when every tool has a typed error shape the model can use to recover, such as {"error":"missing_city"}, while internal exceptions still go to logs.

Hands-On Lab

Prerequisites: Python 3.10 or later and a terminal. No network access or API key is required because the lab simulates the model decision. The goal is to run the dispatch and loop examples, then verify that validation blocks malformed calls.

  1. Create a temporary file named tool_lab.py and paste the validation, dispatch, and fake_model examples into it in that order.
  2. Run python tool_lab.py. You should see the Chicago dictionary and the Seattle final answer.
  3. Add print(dispatch_tool("get_weather", {"city": "Chicago", "admin": True})) at the bottom.
  4. Run the file again. Verification succeeds if Python raises ValueError: unexpected arguments: ['admin'].
  5. Replace the bad call with print(prepare_refund("support_manager", "A100", 1200)) and add the refund example above it. Verification succeeds if output contains needs_confirmation.
  6. Cleanup by deleting tool_lab.py, or keep it as a local test fixture.

Assessment Exercises

  • Design two ecommerce tools: one for checking shipment status and one for cancelling an order. Specify schema fields and which one requires confirmation.
  • A model sends {"order_id":"A100","user_id":"admin"} to a refund tool. Explain which fields should be rejected and where real identity should come from.
  • Given a result containing 200 product records, propose a smaller shape that would still answer, "Which three are cheapest and in stock?"
  • Write a test case proving unknown tools are rejected before any implementation function runs.
  • Explain when parallel tool calls are appropriate and when a sequential loop is safer.

Summary

Tool calling gives generative AI systems controlled access to external data and actions. The model selects a named tool and drafts JSON arguments; your application validates, authorizes, executes, and returns a bounded result. Good tool design uses narrow names, strict schemas, compact outputs, explicit error shapes, and separate confirmation for risky writes.