Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 53 ยท Tools and Agents

Memory, Planning and Context Management

Manage short-term and long-term memory for LLM agents: summarization, scratchpads, context trimming, plan-then-execute and sub-agents. Live Python examples.

A model sees only what is in its context window, and the window is finite. A chatbot that talks for an hour, or an agent that takes thirty tool steps, will eventually produce more text than fits, or more than is sensible to pay for on every request. Someone has to decide what the model still gets to see. That someone is your code, and the techniques are called context management and memory.

This lesson covers the memory types, the common ways to shrink a history, scratchpads for working notes, planning before acting, and splitting work across sub-agents. The examples use simple stand-ins for the model so you can see the mechanics.

Two kinds of memory

Short-term The messages inside the context window.
Long-term Facts saved outside, fetched when needed.

The window is a desk. Long-term memory is the filing cabinet.

1 / 2

Short-term memory: the history

Short-term memory is simply the message list you send on every request. It is perfect recall for as long as it fits, and nothing at all once a message is dropped. Its costs are real: every extra message is paid for again on every later request, long contexts are slower, and very long ones suffer from "lost in the middle" effects (see Context Windows and the KV Cache).

Strategies for keeping it under control:

  • Sliding window: keep the system prompt and the most recent N messages (or the most recent tokens up to a budget). Simple and predictable, but the model forgets early facts such as the user's name or goal.
  • Summarization: when the history passes a threshold, ask a model to summarize the oldest part, then replace those messages with the summary. You keep the gist and pay once for the summary call. Facts can be lost or distorted, so summaries should explicitly preserve names, decisions, numbers and open tasks.
  • Pinning: mark messages that must never be dropped (the system prompt, the goal, safety rules).
  • Tool result clearing: old tool outputs are usually the biggest and least useful part of an agent's history. After the model has used a result, replace it with a one-line note or remove it.

Long-term memory

Long-term memory lives outside the context window: in a database, a file, or a vector store. The agent writes things worth remembering ("user prefers metric units", "project deadline is June 12") and later retrieves the relevant ones, usually with the same retrieval techniques as RAG. Decisions to make:

  • What to save: facts, preferences, decisions and outcomes, not whole transcripts.
  • When to save: after each turn, at the end of a session, or only when the model calls a "remember" tool.
  • How to retrieve: by recency, by similarity to the current question, or both.
  • How to forget: memory needs updates and deletion, because facts change and users may ask to be forgotten. Remember that stored personal data brings privacy obligations (see Bias, Privacy, Copyright and Responsible Use).

A conversation memory manager

The manager below keeps pinned messages, a rolling summary and recent turns inside a token budget. The summarizer is a fake stand-in that just lists the first words of each dropped message; a real one would be a model call with the instruction to keep names, numbers and decisions.

Python Trimming and summarizing a conversation to fit a budget

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

As the budget fills, the oldest turns move into the summary and the newest three stay verbatim. Look carefully at what the toy summary kept: only the first five words of each dropped message. The name "Dana" happened to survive, but the trip length and the month did not. That is the central risk of summarization, and the reason real summarizers are told exactly which kinds of facts to keep. A better design also saves key facts such as "name: Dana, budget: 3000 dollars" to a pinned or long-term store as they appear.

Scratchpads and notes

A scratchpad is a place where the agent writes working notes: partial results, a to-do list, what it has tried. It can be a plain text field that is included in every prompt, or a file the agent reads and writes with tools. Scratchpads help in two ways: the agent does not have to rediscover facts after old tool outputs were trimmed, and its progress is visible to you. Long-running agents often keep a small "state" document (goal, plan, done, next) that is re-injected every turn, which makes the agent robust even when the raw history is cut.

Plan, then execute

Instead of deciding one step at a time, an agent can first write a plan (a numbered list of steps) and then carry the steps out, optionally revising the plan when something unexpected happens. Planning helps with long tasks because it fixes the goal early, makes progress measurable, and lets you show the plan to a user for approval before anything risky runs. The downside is rigidity: if step 2 reveals that step 3 is pointless, a fixed plan wastes effort, so good agents re-plan when observations contradict their assumptions.

Sub-agents

A sub-agent is a separate agent run, with its own fresh context, given one focused task. The parent sends a brief ("find the three most cited papers on topic X, return titles only") and receives a short result. Everything the sub-agent read and tried stays in its own context and is discarded, so the parent's history remains small. Sub-agents are useful for:

  • Isolation: noisy searching or file reading does not pollute the main context.
  • Parallelism: independent tasks can run side by side.
  • Specialization: each sub-agent can have its own prompt, tools and cheaper model.

The cost is overhead and lost detail: the parent only sees what the sub-agent returns, so briefs must be self-contained and results should be compact.

Python Plan-then-execute with sub-agent stand-ins

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The parent only ever sees two short notes, even though the sub-agents did several steps of private work. This is the main reason to split work: you keep the main context small and focused.

When an agent starts behaving oddly late in a long run, suspect the context first: a trimmed fact, a stale tool result, or a summary that dropped something important. Print exactly what the model was sent on the failing step.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the main drawback of a simple sliding window of recent messages?

    A window keeps only the newest messages. Anything important from earlier must be pinned, summarized or stored elsewhere.
  2. 2. Which part of an agent's history is usually the best candidate for clearing first?

    Tool results are big and often useless after the model has extracted what it needed.
  3. 3. Why use a sub-agent for a messy research task?

    Isolation keeps the parent context small. The trade-off is that the brief and the result must carry all needed detail.
  4. 4. What is the main risk of replacing old messages with a model-written summary?

    Summaries are lossy. Tell the summarizer what must be preserved, and keep critical facts in pinned or long-term memory.

Summary

  • Short-term memory is the message history in the context window; long-term memory is stored outside and retrieved when relevant.
  • Control history size with sliding windows, summaries, pinned messages and clearing old tool results.
  • Summaries are lossy: instruct them to keep names, numbers and decisions, and store critical facts separately.
  • Scratchpads and state documents keep an agent oriented after the raw history is cut.
  • Plan-then-execute fixes the goal and makes progress visible; re-plan when observations contradict the plan.
  • Sub-agents isolate noisy work and keep the parent context small; give them self-contained briefs.

Next: see how tools, workflows and several agents fit together in MCP, Workflows and Multi-Agent Systems.