Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 39 ยท Tools and Function Calling

Tool Errors, Retries and Fallbacks

Make agents resilient when tools fail: classify errors, return readable error observations, retry with backoff, add fallbacks and circuit breakers, with Python examples.

Tools fail. Networks drop, APIs rate-limit you, files are missing, and models pass arguments that make no sense. A fragile agent crashes on the first failure. A resilient one treats failure as just another observation: it reads what went wrong, decides whether to retry, change the arguments, try another tool, or give up politely. This lesson shows how to build that behavior around the dispatcher from Executing Tool Calls Safely.

Not all errors are equal

The first step is to classify what happened, because the right reaction depends on the kind of failure.

Kind Example Right reaction
Bad arguments Missing field, wrong type Return the message to the model so it can fix the call
Not found / no result Unknown order id Report it clearly; the model may try a different id
Transient Timeout, HTTP 503, rate limit Retry automatically with a delay
Permanent Permission denied, invalid API key Do not retry; stop or escalate
Unsafe Call blocked by policy Explain the block; never retry the same call

Only the dispatcher knows how an error was produced, so it should convert each failure into a short, structured message. The model is a reader, so give it something it can act on.

flowchart TD
  C["Tool call"] --> R{"Result"}
  R -- "success" --> O["Observation"]
  R -- "bad arguments" --> M["Error message to model"]
  R -- "transient" --> W["Wait, then retry"]
  W --> R
  R -- "permanent" --> E["Stop or escalate"]
  M --> L["Model repairs the call"]

Different failures lead to different next steps.

Errors as readable observations

Never let a Python traceback travel to the model: it is long, leaks internals, and does not say what to do. Instead catch exceptions at one place and return a compact, consistent shape.

Python Turn exceptions into observations

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The hint field is the key idea. A message that says what to try next lets a language model recover in one step instead of guessing.

Retrying transient failures

Transient errors usually disappear if you wait a moment. The standard technique is exponential backoff: wait a short time, then twice as long, then twice again. Add a little random jitter so many clients do not all retry at the same instant. Always cap the number of attempts.

Python Retry with exponential backoff

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The demo prints the wait instead of sleeping, so it runs instantly. In production you would call time.sleep(delay) (or await asyncio.sleep(delay)). If the server tells you how long to wait (a Retry-After header), obey it instead of your own formula.

Only retry operations that are safe to repeat. Reading a record is idempotent; "send an email" or "charge the card" is not. For those, send an idempotency key so the server ignores duplicates, or do not retry automatically at all.

Fallbacks and circuit breakers

When a tool keeps failing, you have two more options.

  • Fallback: try an alternative that gives a similar answer, such as a second weather provider, a cached value, or a simpler search. Tell the model that the answer came from a fallback if freshness matters.
  • Circuit breaker: after several consecutive failures, stop calling that tool for a while. This protects a struggling service and saves the agent from burning its step budget on a dead end.
Python Fallback chain with a circuit breaker

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

By the third query the breaker is open, so the dead service is not even contacted.

Teaching the loop to cope

Even with good dispatcher behavior, the loop needs three rules:

  1. Count errors. If the same tool fails with the same arguments twice, stop repeating it. Models sometimes loop on an identical broken call.
  2. Escalate clearly. After the retry budget is gone, return a plain message to the model ("the order service is unavailable") and let it explain to the user. See stop conditions and budgets.
  3. Log everything. Record every failure with the call, kind and attempt number so you can debug later; see observability and tracing.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why return a short error message with a hint instead of a full traceback?

    A compact, actionable observation helps the model recover and avoids leaking internal details.
  2. 2. Which failure should be retried automatically?

    Transient errors often clear up. The others will fail again until something is changed.
  3. 3. What does jitter add to exponential backoff?

    Without jitter, clients that failed together retry together and overload the service again.
  4. 4. What does an open circuit breaker do?

    It prevents wasting steps and hammering a service that is already struggling.
  5. 5. Why be careful when retrying a tool that sends an email?

    Repeating a side effect can do it twice. Use idempotency keys or require a decision.

Summary

  • Classify failures: bad arguments, not found, transient, permanent, unsafe.
  • Convert exceptions into short, structured observations with a hint.
  • Retry only transient, idempotent operations, with capped exponential backoff and jitter.
  • Use fallbacks and circuit breakers so a dead dependency does not drain the step budget.
  • Count repeated failures, escalate clearly and log every attempt.

Next: Designing Good Tools for Agents