Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 39 ยท Agent Concepts and Loops

Agent State and Message History

Manage what an agent knows: the message transcript, structured state outside the prompt, scratchpads, trimming and summarizing history, and checkpoints.

Because a model call is stateless, an agent's state lives in your program. How you structure that state decides how reliable, cheap and debuggable the agent is. Beginners store everything as a growing list of chat messages. That works for short runs, but longer tasks benefit from splitting state into three layers: the transcript (what was said and done), structured state (facts the program tracks in variables), and the prompt view (the small slice you actually send to the model each turn).

Three layers of state

flowchart TD
  T["Transcript<br/>every message and tool result<br/>(full log, for audit)"] --> V["Prompt view<br/>trimmed, summarized,<br/>relevant slice"]
  S["Structured state<br/>variables: goal, plan,<br/>counters, findings"] --> V
  V --> M["Model call"]

The full log stays on disk; the model sees a curated view assembled from the log and the structured state.

  • Transcript. Append-only record of everything. Never lose it; it is how you debug and evaluate.
  • Structured state. Typed values your code controls: the task, the current plan step, a list of collected results, counters for budgets. These are more reliable than hoping the model remembers a number from 20 messages ago.
  • Prompt view. A function that renders the first two into text for the next call.

A state object

A small data class makes the shape explicit. It also gives you one place to save and restore, which is the foundation of resuming an interrupted run.

Python A structured agent state with a prompt view

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The prompt view lists findings every time, so the model never has to dig them out of the old messages. Even when the oldest messages are dropped, the facts survive in findings. This is a lightweight form of memory, and the memory chapter of this course builds on this idea.

Strategies when history grows

Once the transcript no longer fits comfortably, choose a strategy. Each trades fidelity for size.

  1. Sliding window. Keep the goal plus the last N messages. Simple and predictable, but it forgets older facts.
  2. Summarize old turns. Replace a block of old messages with a short summary written by the model (or by code). Cheaper, but a summary can drop details.
  3. Offload to storage. Save large tool results to a file or database and keep only a handle ("result stored as r7, 4,000 rows, columns: ..."). The agent can fetch parts on demand.
  4. Pin important items. Mark messages such as the goal and user preferences as never removable.
Python Sliding window with pinned messages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pinning is how you make sure a dietary preference or a safety rule is never silently trimmed away.

Checkpoints: saving and resuming

Real tasks get interrupted by crashes, rate limits and deployments. If the state is plain data, you can save it after each step and resume later. Keep it to JSON-friendly types so that saving is trivial.

Python
import json

def save(state, path):
    with open(path, "w") as f:
        json.dump(state.__dict__, f)

def load(path):
    with open(path) as f:
        return AgentState(**json.load(f))

Saving after each completed step (not in the middle of one) gives you at-least-once semantics: a resumed agent may repeat the last step, so tools with side effects should be idempotent or protected by a request ID. Stop conditions and budgets uses the same idea to keep counters across resumes.

Common mistakes

  • Letting the transcript be the only state. Facts become hard to find and impossible to count.
  • Mutating the log when trimming. Trim the view, keep the log.
  • Storing secrets in the transcript. Logs get shared. Keep API keys in the environment, and redact tool output that might contain credentials.
  • Unbounded structured state. A findings list can also grow without limit; cap or summarize it.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the benefit of keeping findings in structured state instead of only in messages?

    Variables under your control are reliable. Facts buried in old messages disappear when history is trimmed.
  2. 2. When compacting history, which messages should be pinned?

    Pinned items must always be present, so they should be the few things the agent can never lose.
  3. 3. Why trim the prompt view rather than the transcript?

    An append-only log is the evidence of what happened; only the slice sent to the model needs to be small.
  4. 4. Why should tools with side effects be idempotent when you resume from checkpoints?

    If a crash happens after a tool ran but before the checkpoint, the step will run again. Idempotency prevents double effects.

Summary

  • State has three layers: transcript, structured state and the prompt view.
  • Keep the log complete; trim only the view sent to the model.
  • Strategies for long runs: sliding windows, summaries, offloading and pinning.
  • Plain-data state can be checkpointed and resumed; make side effects idempotent.
  • Never store secrets in the transcript.

Next: Stop Conditions and Budgets