Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 32 ยท Memory and State

Trimming and Summarizing History

Keep long conversations inside the context window: window and token trimming, trim_messages, rolling summaries and deleting messages. Runnable Python stand-ins included.

If you replay the whole conversation on every call, it grows without limit. Sooner or later it exceeds the model's context window, makes calls slower and more expensive, and buries the useful parts in noise. Long histories also hurt quality: models can miss details in the middle of a very long context. So every serious chatbot needs a policy for what to keep. There are three basic ones: trim old messages, summarize them, or delete specific ones.

LangChain is not available on this site, so real code is read-only. The runnable examples are plain-Python stand-ins for each policy, using a rough word count in place of a real token counter.

Policy 1: keep the last N messages

The simplest policy is a sliding window: keep the system message and the most recent few turns. It is cheap and predictable, but anything older is forgotten completely.

LangChain provides trim_messages for this:

Python
# Real LangChain (read-only)
from langchain_core.messages import trim_messages

trimmed = trim_messages(
    messages,
    strategy="last",          # keep the most recent messages
    max_tokens=1000,          # budget
    token_counter=model,      # use the model's own token counting
    include_system=True,      # never drop the system message
    start_on="human",         # the result must begin with a human message
)

Details that matter:

  • include_system=True keeps your instructions even when old turns are dropped.
  • start_on="human" avoids a history that begins with an orphan AIMessage or ToolMessage, which some providers reject.
  • A tool call and its result must stay together. Dropping one without the other breaks the conversation, so trim on message boundaries that keep these pairs.

Instead of counting tokens you may pass token_counter=len and a max_tokens that means "number of messages".

Python Stand-in: a sliding window that keeps the system message

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the first user question was dropped but the system message survived, and the window starts on a user turn.

Policy 2: summarize old messages

Dropping history loses facts ("my budget is 500 euros"). A rolling summary replaces old messages with a short note written by the model, then keeps recent messages verbatim. The cost is one extra model call now and then, and the risk that the summary leaves something out.

Python
# Real LangChain (read-only): summarization as agent middleware
from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware

agent = create_agent(
    model="anthropic:claude-sonnet-4-5",
    tools=[],
    middleware=[
        SummarizationMiddleware(
            model="anthropic:claude-haiku-4-5",   # a cheaper model writes the summary
            max_tokens_before_summary=4000,        # trigger
            messages_to_keep=20,                   # keep the latest messages as they are
        )
    ],
)

Parameter names may differ between releases, so check the middleware documentation for your installed version. The shape is stable: a trigger size, a model that writes the summary, and a number of recent messages to keep.

The stand-in below uses a simple rule-based "summarizer" where a real system would call a model:

Python Stand-in: rolling summary plus recent messages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The budget survived inside the summary even though the original message is gone.

Policy 3: delete or filter

Some messages are pure noise: huge tool outputs, retries, debug text. Instead of trimming by age, remove them by content. LangGraph lets a node return RemoveMessage(id=...) objects to delete particular messages from state, and you can shorten big tool results before they enter the history at all (store the full text elsewhere and keep a snippet plus a reference).

Python
# Real LangChain (read-only)
from langchain_core.messages import RemoveMessage

def drop_old_tool_messages(state):
    old = [m for m in state["messages"][:-6] if m.type == "tool"]
    return {"messages": [RemoveMessage(id=m.id) for m in old]}

Choosing a policy

Policy Strength Weakness
Window Simple, predictable cost Forgets early facts
Token trim Fits any budget exactly Same forgetting
Summary Keeps key facts compactly Extra call, lossy
Delete noise Removes junk, keeps real turns Needs rules per message type
Long-term store Facts persist across threads More moving parts (next lessons)

Many apps combine them: summarize past a threshold, always keep the latest turns, and move durable facts (name, preferences) into a separate store.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does trim_messages offer include_system=True?

    The system message defines behavior; losing it with old messages would change the assistant.
  2. 2. What is the main downside of a rolling summary?

    A summary may omit details and requires a model call to write it.
  3. 3. What must be kept together when trimming an agent conversation?

    Providers reject a ToolMessage without its matching call, or a call without its result.
  4. 4. What does a sliding window forget?

    Anything older than the window is gone unless stored elsewhere.

Summary

  • Replaying everything grows cost and hurts quality, so choose a history policy.
  • trim_messages keeps the last messages within a budget; keep the system message and valid message boundaries.
  • Summaries preserve facts compactly but are lossy and cost a call.
  • Delete noisy messages by rule; store durable facts outside the conversation.

Next: Persistent Checkpoints and Long-Term Memory