Tool Errors, Retries and Timeouts

A tool is code that an agent calls to touch the real world: a web API, a database, a file. Real systems fail. Networks drop, servers say “try later”, and some calls hang forever. This lesson shows how to turn those failures into clear results, retry only when it is safe, and stop slow calls with timeouts, so your agent stays reliable instead of crashing or looping.

Overview: why tool failures need special care

In normal programs, an exception goes up the call stack to a developer who reads the traceback. In an agent, the reader of the failure is often a language model (LLM). The model sees only the text you put in the tool result. If you crash the whole run, the user gets nothing. If you return a vague message such as “error”, the model guesses, and it may guess wrong, for example by repeating the same bad call ten times.

So every tool call should pass through a thin layer that does three jobs:

1. Catch exceptions and convert them into a structured result (ok or not, message, whether retrying could help).
2. Retry automatically when the failure is transient (temporary), using waiting time between attempts.
3. Limit time with a timeout, so one hung call cannot freeze the agent.

Two kinds of failure

• Transient errors go away if you wait and try again: a network timeout, HTTP 429 (too many requests), HTTP 503 (service unavailable). Retrying is reasonable.
• Permanent errors will fail every time: invalid input, HTTP 404 (not found), HTTP 403 (forbidden), a bug. Retrying wastes time and can cause harm. These should be sent back to the model (for input mistakes it can fix) or to the user.

The most important skill in this lesson is classifying an error as transient or permanent. Everything else follows from that.

Who handles what

There are three layers. The tool wrapper handles timeouts and automatic retries for transient faults; the model never needs to see these. The model handles errors that need reasoning, such as “the date format was wrong, let me fix it”. The agent loop handles global limits, such as a maximum number of steps, and asks a human when something risky fails. Keeping these layers separate makes agents easier to debug.

Key concepts and general form

A good tool result is a small dictionary with the same shape every time. The model learns the shape quickly.

{
  "ok": false,
  "error": "city must not be empty",
  "retryable": false
}

Examples

Example 1: turn exceptions into structured results

We define our own exception classes. Each class says whether retrying is useful. The function run_tool catches everything and always returns a dictionary.

class ToolError(Exception):
    retryable = False


class TransientError(ToolError):
    retryable = True


class InvalidInputError(ToolError):
    retryable = False


def get_weather(city):
    if not city.strip():
        raise InvalidInputError('city must not be empty')
    if city.lower() == 'flaky':
        raise TransientError('weather service returned 503')
    return {'city': city, 'temp_c': 31}


def run_tool(fn, **kwargs):
    try:
        return {'ok': True, 'result': fn(**kwargs)}
    except ToolError as exc:
        return {'ok': False, 'error': str(exc), 'retryable': exc.retryable}
    except Exception as exc:
        return {'ok': False, 'error': 'unexpected ' + type(exc).__name__, 'retryable': False}


print(run_tool(get_weather, city='Pune'))
print(run_tool(get_weather, city=' '))
print(run_tool(get_weather, city='flaky'))
print(run_tool(get_weather))

Output:

{'ok': True, 'result': {'city': 'Pune', 'temp_c': 31}}
{'ok': False, 'error': 'city must not be empty', 'retryable': False}
{'ok': False, 'error': 'weather service returned 503', 'retryable': True}
{'ok': False, 'error': 'unexpected TypeError', 'retryable': False}

The last call forgot the city argument, so Python raised TypeError. The generic branch catches it. Notice that for unknown errors we return only the exception type, not the full message or traceback. Raw exceptions can contain file paths, URLs with tokens, or database details, and you should not hand those to a model or a user.

Example 2: retry with exponential backoff

Exponential backoff doubles the wait after every failure, up to a cap. The helper below takes an injectable sleep function so the example runs instantly. In production you would leave the default time.sleep.

import random
import time


class TransientError(Exception):
    pass


def backoff_delay(attempt, base=0.5, cap=8.0, jitter=False):
    delay = min(cap, base * 2 ** (attempt - 1))
    return random.uniform(0, delay) if jitter else delay


def call_with_retry(fn, max_attempts=4, sleep=time.sleep):
    for attempt in range(1, max_attempts + 1):
        try:
            return fn()
        except TransientError as exc:
            if attempt == max_attempts:
                raise
            delay = backoff_delay(attempt)
            print(f'attempt {attempt} failed ({exc}); retrying in {delay}s')
            sleep(delay)


calls = {'n': 0}


def flaky():
    calls['n'] += 1
    if calls['n'] < 3:
        raise TransientError('503')
    return 'ok after %d calls' % calls['n']


print(call_with_retry(flaky, sleep=lambda s: None))

Output:

attempt 1 failed (503); retrying in 0.5s
attempt 2 failed (503); retrying in 1.0s
ok after 3 calls

Only TransientError is caught. Any other exception passes straight through, which is correct: a bug should not be retried. On the final attempt we re-raise, so the caller still learns that the tool failed. In real systems set jitter=True. Without it, 1000 agents that failed together will all retry together and hit the struggling server in waves.

Example 3: a timeout around a slow tool

Python cannot safely kill a running function. A common pattern is to run the tool in a worker thread and stop waiting after a deadline.

import time
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FutureTimeout

_pool = ThreadPoolExecutor(max_workers=4)


def run_with_timeout(fn, timeout_s, **kwargs):
    future = _pool.submit(fn, **kwargs)
    try:
        return {'ok': True, 'result': future.result(timeout=timeout_s)}
    except FutureTimeout:
        future.cancel()
        return {'ok': False, 'error': f'tool timed out after {timeout_s}s', 'retryable': True}
    except Exception as exc:
        return {'ok': False, 'error': str(exc), 'retryable': False}


def slow_search(query):
    time.sleep(0.5)
    return f'results for {query}'


print(run_with_timeout(slow_search, 0.1, query='mcp'))
print(run_with_timeout(slow_search, 2.0, query='mcp'))
_pool.shutdown(wait=True)

Output:

{'ok': False, 'error': 'tool timed out after 0.1s', 'retryable': True}
{'ok': True, 'result': 'results for mcp'}

The first call needs 0.5 seconds but we allow 0.1, so we report a timeout. Important: the worker thread keeps running until slow_search finishes. future.cancel() only works if the job has not started yet. For network calls, the better fix is to also pass a timeout to the client itself (most HTTP libraries accept one), so the work really stops.

Example 4: the agent loop recovers from an error

Here a scripted call_llm() stub plays the model, so the example runs without network access. To use a real model, replace the stub with a function that sends messages to your provider’s SDK and converts the reply into the same two shapes (tool_call or final). The first call has an empty city. The model sees the error and corrects itself.

class ToolError(Exception):
    retryable = False


class InvalidInputError(ToolError):
    retryable = False


def get_weather(city):
    if not city.strip():
        raise InvalidInputError('city must not be empty')
    return {'city': city, 'temp_c': 31}


def run_tool(fn, **kwargs):
    try:
        return {'ok': True, 'result': fn(**kwargs)}
    except ToolError as exc:
        return {'ok': False, 'error': str(exc), 'retryable': exc.retryable}
    except Exception as exc:
        return {'ok': False, 'error': 'unexpected ' + type(exc).__name__, 'retryable': False}


def call_llm(messages):
    # Stub: replace with a real SDK call. Scripted for this demo.
    last = messages[-1]
    if last['role'] == 'user':
        return {'type': 'tool_call', 'name': 'get_weather', 'args': {'city': ''}}
    outcome = last['content']
    if not outcome['ok'] and outcome['error'].startswith('city must'):
        return {'type': 'tool_call', 'name': 'get_weather', 'args': {'city': 'Pune'}}
    if outcome['ok']:
        temp = outcome['result']['temp_c']
        return {'type': 'final', 'text': f'It is {temp} C in Pune.'}
    return {'type': 'final', 'text': 'Sorry, I could not get the weather.'}


TOOLS = {'get_weather': get_weather}


def agent(question, max_steps=5):
    messages = [{'role': 'user', 'content': question}]
    for step in range(max_steps):
        reply = call_llm(messages)
        if reply['type'] == 'final':
            return reply['text']
        outcome = run_tool(TOOLS[reply['name']], **reply['args'])
        print(f'step {step}: {reply["name"]} -> {outcome}')
        messages.append({'role': 'tool', 'content': outcome})
    return 'Stopped: step limit reached.'


print(agent('Weather in Pune?'))

Output:

step 0: get_weather -> {'ok': False, 'error': 'city must not be empty', 'retryable': False}
step 1: get_weather -> {'ok': True, 'result': {'city': 'Pune', 'temp_c': 31}}
It is 31 C in Pune.

Two details matter. The error text told the model exactly what to fix. And max_steps guarantees the loop ends even if the model never succeeds.

Under the hood: the life of one failing call

1. The model asks for a tool call with arguments.
2. Your code validates the arguments. Bad input becomes an InvalidInputError immediately, with no network call.
3. The wrapper starts the call with a timeout.
4. If a transient fault occurs, the wrapper waits (backoff plus jitter) and tries again, up to a fixed number of attempts.
5. If it still fails, the wrapper returns a structured error result. It does not raise.
6. The loop appends that result to the conversation. The model reads it and chooses: fix the input, try another tool, ask the user, or give up politely.

Watch out for multiplied retries. If the wrapper tries 4 times and the HTTP client inside also retries 3 times, one call can become 12 requests. Retry at one layer only. Also respect the Retry-After header when a server sends it with a 429 or 503; it tells you the minimum wait.

Errors in MCP servers

If you build tools with the official MCP Python SDK, the same ideas apply. A tool function on a FastMCP server that raises an exception is reported to the client as a tool error result rather than crashing the server. Raise clear, safe messages.

from mcp.server.fastmcp import FastMCP

mcp = FastMCP('weather')


@mcp.tool()
def get_weather(city: str) -> str:
    """Return the current temperature for a city."""
    if not city.strip():
        raise ValueError('city must not be empty')
    return city + ': 31 C'


if __name__ == '__main__':
    mcp.run()

This needs the mcp package installed, so it is shown as a fragment. Add timeouts inside slow tools, because the client’s patience is not under your control.

Common Mistakes

Mistake 1: retrying everything, including actions with side effects

def send_email_tool(to, body):
    return smtp.send(to, body)

for attempt in range(5):
    try:
        result = send_email_tool('a@example.com', 'Hi')
        break
    except Exception:
        pass

This is wrong for two reasons. It swallows every error, including bugs. And if the email was sent but the reply was lost (a timeout after success), the loop sends it again, up to five times. The fix is an idempotency key: a unique ID sent with the request, so the receiver can detect duplicates. Many payment and messaging APIs support this. The simulation below shows the idea.

import uuid

SENT = {}


def send_email_tool(to, body, idempotency_key):
    if idempotency_key in SENT:
        return SENT[idempotency_key]
    SENT[idempotency_key] = 'sent to ' + to
    return SENT[idempotency_key]


key = str(uuid.uuid4())
print(send_email_tool('a@example.com', 'Hi', key))
print(send_email_tool('a@example.com', 'Hi', key))

Output:

sent to a@example.com
sent to a@example.com

The second call is a safe repeat: it returns the stored result and sends nothing new. Generate the key once per intended action, before the first attempt.

Mistake 2: a with block that silently removes the timeout

with ThreadPoolExecutor() as pool:
    future = pool.submit(slow_search, 'mcp')
    future.result(timeout=0.1)

result(timeout=0.1) does raise a timeout error. But leaving the with block calls shutdown(wait=True), which waits for the slow thread to finish. Your “timeout” still blocks for the full duration. Keep a long-lived pool (as in Example 3) or call shutdown(wait=False).

Mistake 3: leaking raw errors to the model

Returning str(exc) or a full traceback for unknown errors can expose secrets and internal paths. It also gives an attacker, or a prompt-injected web page, more to work with. Return a short safe message and write the details to your own logs.

Best Practices

• Always return one consistent result shape: ok, result or error, and retryable.
• Write error messages for the model: say what was wrong and what a valid value looks like.
• Retry only transient errors and only idempotent calls. Cap attempts (3 to 5 is typical) and cap the total wait.
• Use exponential backoff with jitter, and honour Retry-After.
• Set a timeout on every tool and on the underlying network client. Choose different values for fast lookups and slow jobs.
• Cap the agent loop with a maximum number of steps, and also cap repeated identical failing calls.
• Require human approval before retrying or re-running risky actions (payments, deletions, outgoing messages), especially after an ambiguous failure such as a timeout.
• Treat tool output as untrusted data. An error message that comes from a web page or third-party API might contain instructions aimed at the model (prompt injection). Never let it change permissions or trigger actions on its own.
• Log every attempt with the tool name, attempt number and duration, but never log secrets.

Practice Exercises

1. Classify HTTP status codes. Write is_retryable(status) that returns True for 429, 500, 502, 503 and 504, and False for other codes such as 400, 401, 403 and 404. Test it on those values. Hint: a set makes this one line.
2. Retry budget. Extend call_with_retry from Example 2 with a max_total_wait argument. Stop retrying (and re-raise) if the sum of delays would exceed it.
3. Repeat-failure guard. In the agent loop of Example 4, remember each (tool name, arguments) pair that failed with retryable false. If the model asks for the same pair again, skip the call and return an error saying it already failed. Hint: use json.dumps(args, sort_keys=True) to build a hashable key.

Summary

• Wrap every tool call so failures become structured results, not crashes.
• Classify errors: transient ones may be retried, permanent ones go back to the model or the user.
• Use exponential backoff with jitter, a small attempt limit, and retry at a single layer.
• Timeouts protect the loop, but threads keep running; set client-level timeouts too.
• Never blindly retry actions with side effects; use idempotency keys and human approval.
• Keep error text useful but safe, and treat all tool output as untrusted input.
• Always bound the agent loop with a step limit.