Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 38 ยท Machine Learning Apps

Chat Apps with st.chat_message

Build chat interfaces in Streamlit with st.chat_message and st.chat_input. Keep history in session state, stream replies and call a language model API safely.

Chat is now a standard way to put a model in front of users, and Streamlit has widgets made for it: st.chat_input for the message box, st.chat_message for each bubble and st.write_stream for text that appears as it is generated. The tricky part is not the widgets. It is the rerun model: when the user sends a message, the whole script runs again, so the conversation must be stored in st.session_state and redrawn each time.

This lesson builds a chat app in three steps, runs the conversation logic with a stand-in bot, and then shows where a real language model API plugs in.

Streamlit and network calls do not run on this site. Chat widget code is read-only. The runnable examples implement the message history and a rule-based reply function in plain Python, which is the same logic the app uses, and the API call is shown read-only because we cannot make a real network request here.

The chat widgets

Python
import streamlit as st

st.title("Echo chat")

if "messages" not in st.session_state:
    st.session_state.messages = []            # list of {"role": ..., "content": ...}

for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

if prompt := st.chat_input("Say something"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    reply = f"You said: {prompt}"
    st.session_state.messages.append({"role": "assistant", "content": reply})
    with st.chat_message("assistant"):
        st.markdown(reply)

Four ideas carry the whole pattern:

  1. History lives in session state as a list of dictionaries with role ("user" or "assistant") and content. This is also the format most model APIs expect, so you can pass it straight along.
  2. Replay on every run. The loop at the top redraws all earlier bubbles, because the script starts from scratch each time (see How Streamlit Reruns Your Script).
  3. st.chat_input returns None until the user sends something. The walrus operator (:=) assigns and tests in one line. It is pinned to the bottom of the page automatically.
  4. Draw the new messages immediately. The history loop already ran before the new message existed, so the new pair is drawn once directly, and appears in the loop on the next run.

st.chat_message takes "user", "assistant", or any string name (with an optional avatar="๐Ÿค–" emoji or image). Inside it you can put anything: st.markdown, st.dataframe, st.image, st.code, charts, even other widgets.

The conversation logic, runnable

The core of a chat app is a function reply(history) that takes the whole history and returns a string. Here is one that needs no network: it answers a few questions from rules, remembers the user's name from earlier in the conversation, and keeps a transcript just like session state would.

Python Stand-in: history plus a rule-based assistant

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because the assistant sees the whole history, it can answer "what is my name?" two turns after being told. Real language models work the same way: they have no memory of their own, you send the conversation each time.

Calling a language model (read-only)

Replace the reply stub with an API call. The shape is the same for most providers: send the list of messages, receive text. The example below uses a generic pattern; check your provider's documentation for the exact client and model names.

Python
import streamlit as st
from my_llm_client import Client             # your provider's official SDK

client = Client(api_key=st.secrets["LLM_API_KEY"])   # never hard-code keys

SYSTEM = "You are a concise assistant for a tutorials website."

def generate(messages):
    response = client.chat(
        model="your-model-name",
        system=SYSTEM,
        messages=messages,
        max_tokens=500,
    )
    return response.text

Remember the lessons from APIs and Secrets: the key comes from st.secrets, set a timeout and an output limit, and handle errors with try/except and st.error so a failed call does not leave a half-drawn page.

Streaming

Waiting ten seconds for a full answer feels broken. Streaming shows words as they arrive. st.write_stream accepts a generator and returns the full text once it finishes:

Python
def stream_reply(messages):
    for chunk in client.chat_stream(model="your-model-name", messages=messages):
        yield chunk.text

with st.chat_message("assistant"):
    full_text = st.write_stream(stream_reply(st.session_state.messages))
st.session_state.messages.append({"role": "assistant", "content": full_text})

Save the returned full_text to the history, otherwise the answer vanishes on the next rerun. A generator is also easy to test, as this stand-in shows: it yields a reply word by word and the joined result matches what write_stream would return.

Python Stand-in: a streaming generator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Controlling cost and context

Every call sends the whole history, so long chats get slow and expensive, and eventually exceed the model's context limit. Three habits keep this under control:

  • Trim the history. Send only the last N messages, for example messages[-20:], or summarize old ones.
  • Cap output. Use a max_tokens setting and ask the model to be brief in the system prompt.
  • Rate-limit users. Count requests in session state and stop after a limit, so one visitor cannot spend your whole budget. Better still, require a login for a public app.
Python
MAX_MESSAGES = 20
if len([m for m in st.session_state.messages if m["role"] == "user"]) >= MAX_MESSAGES:
    st.warning("Message limit reached for this session.")
    st.stop()

Session limits are easy to bypass by refreshing, so also set spending limits at your provider.

A "Clear chat" button

Python
if st.sidebar.button("Clear chat"):
    st.session_state.messages = []
    st.rerun()

Without the st.rerun() the old bubbles would still be on screen for one more run.

Safety and honesty

  • Treat model output as untrusted text. Render it with st.markdown (which does not run scripts by default). Avoid unsafe_allow_html=True on model output.
  • Prompt injection. If your app pastes web pages, documents or user data into the prompt, hidden instructions in that content can steer the model. Do not give the model access to secrets or powerful actions based on untrusted text.
  • Do not log private conversations without telling users.
  • Say it is an AI. A short caption such as "Answers are generated and may contain mistakes" sets expectations.

Common mistakes

  • Forgetting to append the assistant's reply to the history, so it disappears on the next run.
  • Not replaying history at the top of the script, so earlier messages vanish.
  • Putting the API key in the source code.
  • Sending an ever-growing history without trimming.
  • Rendering model output as raw HTML.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why must a Streamlit chat app store messages in st.session_state?

    Each user message triggers a rerun from the top. Session state keeps the conversation between runs so it can be redrawn.
  2. 2. What does st.chat_input return when the user has not sent anything?

    It returns None until a message is sent, which is why if prompt := st.chat_input(...) works.
  3. 3. After st.write_stream finishes, what should you do with its return value?

    The returned full text must be stored, otherwise the assistant's answer is not part of the history and disappears on the next rerun.
  4. 4. Which is a good way to limit cost in a chat app?

    Trimming history, capping output and rate limiting keep both token use and spending predictable.

Summary

  • Keep the conversation as a list of {"role", "content"} dictionaries in st.session_state and replay it on every run.
  • st.chat_input returns None until a message is sent; st.chat_message draws one bubble.
  • Model APIs are stateless: send the history each time, and append the reply to it.
  • st.write_stream shows streamed text and returns the full string to store.
  • Keep keys in st.secrets, trim history, cap output and treat model output and external text as untrusted.

Next: Configuration and Themes