Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 37 ยท Projects and Practice

Project: Chat Assistant with Memory

Build a chat assistant with Gradio Blocks: per-session memory with gr.State, slash commands, history trimming, streaming replies and a pluggable model, plus practice exercises.

Your last project is a chat assistant that remembers facts the user tells it, understands a few slash commands, trims long conversations and streams its replies. It uses a rule-based "brain" so the logic runs in this lesson; the brain is a single function you can replace with a real language model.

You could build much of this with gr.ChatInterface (see chatbots), but this version uses gr.Blocks, gr.Chatbot and gr.State, so you control the layout and keep per-user memory. The Gradio code is real and read-only (the library does not run on this site); the logic is plain Python and runnable.

Design

flowchart TB
    U["User message"] --> C{"Starts with /?"}
    C -- yes --> CMD["run_command()<br/>/remember, /forget, /memory, /help"]
    C -- no --> BR["brain(message, memory, history)"]
    CMD --> R["reply"]
    BR --> R
    R --> S["stream words to Chatbot"]
    S --> ST["update gr.State memory"]

Commands and normal messages share one reply path, so streaming and history work the same for both.

Step 1: memory and commands

Memory is a dictionary of facts, kept per user session in gr.State (never in a module-level variable, which all visitors would share).

Python Commands and memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The output shows each command answered in turn: two facts stored, a two-line memory listing, one forgotten, an apology for an unknown key, a usage hint and an unknown-command message with the help text. Returning a new dictionary instead of changing the old one matters for gr.State, because Gradio compares and copies state between calls.

Step 2: the brain

The brain gets the message, memory and history. This one is deliberately simple; it uses memory to personalize replies and the history to count turns.

Python A rule-based brain that uses memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You see a greeting using the name Ada, a correct recall of green, a polite "I do not know your pet", the fallback for an unanswerable question and an echo. To use a real model, send memory as a system prompt line ("The user's name is Ada...") and history as the message list, as in the chatbot lesson.

Step 3: trimming and streaming

Keep the last ten messages, and stream the reply word by word. The generator yields the whole text so far each time:

Python Trim history and stream a reply

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Only the last six messages remain (m4 to m9), and the reply grows word by word over six updates. In the UI the bot bubble is replaced with each value.

Step 4: the Gradio app

The event handler is a generator that updates the chat and state. It uses the message format with role and content:

Python
import gradio as gr
from core import run_command, brain, trim, stream

def chat(message, history, memory):
    if not message.strip():
        raise gr.Error("Please type a message.")
    history = history + [{"role": "user", "content": message}]
    if message.startswith("/"):
        reply, memory = run_command(message, memory)
    else:
        reply = brain(message, memory, trim(history[:-1]))
    history.append({"role": "assistant", "content": ""})
    for partial in stream(reply):
        history[-1]["content"] = partial
        yield history, memory, ""          # third value clears the textbox

with gr.Blocks(title="Chat assistant") as demo:
    gr.Markdown("# Assistant with memory\nTry `/help`.")
    memory = gr.State({})                  # one dict per browser session
    bot = gr.Chatbot(type="messages", height=380)
    box = gr.Textbox(placeholder="Type a message and press Enter", show_label=False)
    with gr.Row():
        clear = gr.Button("Clear chat")
    box.submit(chat, [box, bot, memory], [bot, memory, box])
    clear.click(lambda: ([], {}), None, [bot, memory])

demo.launch()

After launch, typing /remember name=Ada produces a confirmation; later "hello" answers "Hello, Ada!". Each browser tab gets its own gr.State, so two visitors never see each other's memory. The Clear chat button resets both the chat and the memory. Note that gr.State lives on the server while the page stays open; for memory that survives a reload, save to a database keyed by a user name or login (see authentication).

Testing and shipping

Tests for run_command, brain, trim and stream need no Gradio at all: assert on returned replies and on memory before and after (that run_command leaves its input untouched is worth an explicit test). Add one gradio_client test of the endpoint. For deployment, follow Spaces; if you add a hosted model, store the key as a secret and cap the length of both the input and the history, as the errors and performance lesson explains.

Practice exercises

  1. Add a /reset command that clears only the history count (store the turn count in memory).
  2. Make /remember accept several pairs: /remember name=Ada color=green.
  3. Write a test that /forget on an unknown key leaves memory unchanged and returns a helpful message.
  4. Show the current memory in a gr.JSON component next to the chat.
  5. Replace brain with a function that builds an OpenAI-style message list with the memory in a system message, and test it with a fake client.
  6. Add concurrency_limit=2 and max_size=10 to the chat event and write down what visitors see when the queue is full.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is memory kept in gr.State and not a global variable?

    gr.State holds a separate value for each browser session.
  2. 2. Why does run_command return a new dictionary?

    Working on a copy keeps state updates explicit and easy to test.
  3. 3. In the chat handler, what does yielding a third empty string do?

    The values map to the outputs [bot, memory, box]; an empty string clears box.
  4. 4. What is the simplest way to keep prompts short in a long chat?

    Trimming bounds the context size; smarter versions summarize older turns.

Summary

  • Keep per-user memory in gr.State; return new values rather than mutating shared ones.
  • Route slash commands and normal messages through one reply path.
  • Trim history, stream replies by yielding the growing text, and clear the box with an extra output.
  • Keep logic in plain functions so you can test and replace the brain.

Back to the Gradio track page to review all chapters.

Next: Gradio overview