Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 38 ยท Structured Output and Tool Use

The Tool-Calling Loop

Build the loop behind assistants that use tools: send messages, run requested calls, feed results back and stop safely. Runnable simulation with a scripted model.

A single tool call is rarely the whole story. A question like "What is the weather in Oslo and Cairo, and which is warmer?" needs two lookups and a comparison. The pattern that handles this is the tool-calling loop: keep calling the model, run whatever tools it requests, append the results, and stop when the model gives a final answer.

This lesson builds that loop in plain Python with a scripted stand-in model, so you can run and inspect every step without any API. A real model replaces the stand-in; the loop does not change.

The loop

flowchart TB
  S["Start: messages = system + user"] --> C["Call the model"]
  C --> D{"Did it request<br/>tool calls?"}
  D -->|"yes"| R["Run each call<br/>(validate first)"]
  R --> A["Append results<br/>to messages"]
  A --> L{"Step limit<br/>reached?"}
  L -->|"no"| C
  L -->|"yes"| X["Stop: report failure"]
  D -->|"no"| F["Final answer: return it"]

The model decides when it is done; your code decides when to give up.

Four rules keep the loop safe:

  1. Always cap the steps. A confused model could call tools forever.
  2. Validate every call before running it (previous lesson).
  3. Return errors as results, not exceptions, so the model can recover.
  4. Keep the full message list, including each call and its result, so the model sees what has already happened.

A scripted stand-in model

The stand-in below plays a fixed script: first it requests two weather lookups, then (after seeing results) it writes the final answer. It is not intelligent; it exists to make the loop runnable.

Python A complete tool-calling loop (scripted model)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the printed trace top to bottom. It is exactly what you would log in production: which step, which calls, which results. When an assistant misbehaves, this trace is the first place to look.

The step limit in action

To see why the cap matters, here is a stand-in model that never stops asking for tools. The loop's limit ends it safely.

Python A runaway model is stopped by the step limit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Handling errors well

A tool error is not a crash. Return it as a result so the model can adjust:

JSON
{"error": "unknown city: Oslo2. Did you mean Oslo?"}

A helpful error message names the problem and, if possible, the fix. The model may then retry with corrected arguments, ask the user a clarifying question, or explain the limitation. Compare with an uncaught exception, which ends the whole request.

Cost and latency

Each trip around the loop is a full model call that re-reads the entire message list. So:

  • More steps means more cost and more waiting.
  • Results can be long. Trim or summarize large tool outputs before sending them back.
  • Independent calls requested together can run in parallel in your code.
  • Cache results of deterministic tools (same arguments, same answer).
Python Trim a large tool result before returning it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For structured data, a better approach than slicing text is to return only the fields the model needs, or a summary plus a way to fetch more.

Where loops go from here

When the loop gains planning, memory and several cooperating tools, it is called an agent. The building blocks are the ones you now have: messages, validated tool calls, a result history and hard limits. The track on AI agents goes deeper. Later chapters of this course return to loops from the angles of retrieval, safety and production reliability.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. When does the tool-calling loop end normally?

    The model decides it is done by answering without requesting more tools.
  2. 2. Why must the loop have a step limit?

    A hard cap controls cost and prevents runaway loops.
  3. 3. A tool fails. What should your code do?

    Error results let the model retry, ask the user or explain the limitation.
  4. 4. What should you do with very large tool outputs?

    Big results increase cost and can crowd out useful context.
  5. 5. What is a good first place to look when an assistant misbehaves?

    A step-by-step trace shows what the model asked for and what it received.

Summary

  • The tool-calling loop: call the model, run requested tools, append results, repeat until a final answer.
  • Always cap steps, validate calls, return errors as results and keep the full message history.
  • Trim large tool outputs and cache deterministic results to control cost and latency.
  • Log a trace of every step; agents are loops with more planning and memory built on the same pieces.

Next: The Context Window and Token Budgets