Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 38 ยท Data Connections

Web APIs and Secrets in Streamlit

Call web APIs from Streamlit with requests, cache responses, handle errors and timeouts, and keep API keys safe with st.secrets and secrets.toml.

Many dashboards pull live data from web APIs: weather, exchange rates, GitHub, a company's internal service. The pattern is always the same: call the API, check the response, turn the JSON into a DataFrame, show it. With Streamlit there are three extra concerns. Your script reruns on every click, so uncached calls would hammer the API and may break rate limits. API keys must stay out of your source code. And network calls fail, so the page needs to handle that gracefully.

Streamlit and real network calls do not run on this site. App code is read-only. The runnable examples use a fake API function that returns JSON text, so the parsing, error handling and caching logic you test here is real Python.

Calling an API with requests

Python
import pandas as pd
import requests
import streamlit as st

@st.cache_data(ttl=600, show_spinner="Fetching rates...")
def get_rates(base: str) -> pd.DataFrame:
    response = requests.get(
        "https://api.example.com/v1/rates",
        params={"base": base},
        timeout=10,
    )
    response.raise_for_status()                    # raises on 4xx / 5xx
    data = response.json()
    return pd.DataFrame(
        {"currency": list(data["rates"]), "rate": list(data["rates"].values())}
    )

base = st.selectbox("Base currency", ["USD", "EUR", "GBP"])
try:
    st.dataframe(get_rates(base))
except requests.RequestException as error:
    st.error(f"Could not load rates: {error}")

Four habits are visible here and are worth making automatic:

  1. timeout=. Without it a stuck server freezes your page forever. Always set one (5 to 15 seconds).
  2. raise_for_status(). Turns HTTP errors into exceptions so you do not parse an error page as data.
  3. @st.cache_data(ttl=...). One call per base currency per ten minutes, however many reruns happen. See Caching.
  4. try/except around the call site, not inside the cached function. An exception raised inside a cached function is not cached, so the next rerun tries again, which is what you want after a temporary failure. Showing the message with st.error keeps the page alive.

Pass the user's choices as parameters (params=) rather than building URLs by hand, so special characters are escaped properly.

Practice parsing and errors offline

The next TryIt replaces requests with a fake fake_get function that returns canned responses, including a failing one. The conversion and error handling code is the same you would use for a real API.

Python Stand-in: parse JSON and handle failures

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The good call becomes a clean table and the bad one produces a readable message instead of a crash.

Rate limits and the cache

Most free APIs allow a limited number of calls per minute. If your page runs get_rates uncached, a user who drags a slider ten times makes ten calls. Caching by argument means each distinct input calls the API once per ttl. This stand-in counts API calls with and without a cache.

Python Stand-in: a cache protects the API quota

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Five reruns cost five calls without a cache; six reruns cost two with one. st.cache_data(ttl=...) adds expiry on top of this idea.

Secrets: keeping keys out of your code

API keys, database passwords and tokens never belong in your source file. Streamlit reads them from .streamlit/secrets.toml, a TOML file next to your app, and exposes them as st.secrets:

TOML
# .streamlit/secrets.toml  (add this file to .gitignore!)
weather_api_key = "abc123-not-a-real-key"

[database]
host = "db.example.com"
user = "reader"
password = "s3cret"
Python
import streamlit as st

key = st.secrets["weather_api_key"]
db_host = st.secrets["database"]["host"]      # sections become nested dictionaries

response = requests.get(url, params={"q": city, "appid": key}, timeout=10)

Rules that keep you out of trouble:

  • Add .streamlit/secrets.toml to .gitignore before the first commit. A key that reached a public repository must be considered leaked and rotated.
  • On a deployed app, paste the same TOML into the host's secrets setting instead of uploading the file (see Deploying to Streamlit Community Cloud).
  • Never print secrets, never put them in st.write for debugging, and never include them in a cache key or an error message shown to users.
  • Use read-only or scoped keys when the provider offers them.
  • Secrets are also available as environment variables if you prefer (os.environ), which suits Docker deployments.

Missing secrets raise a KeyError or StreamlitSecretNotFoundError. A friendly check at the top of the app gives a better message:

Python
if "weather_api_key" not in st.secrets:
    st.error("Set weather_api_key in .streamlit/secrets.toml to use this app.")
    st.stop()

Asking users for their own key

Some apps let each user bring their own key, for instance for an AI service. Use st.text_input(..., type="password") and keep it in session state only. Never store it in a cache or log it:

Python
api_key = st.text_input("Your API key", type="password")
if not api_key:
    st.info("Enter your key to continue.")
    st.stop()

The key is sent to your server, so only do this over HTTPS and be clear in the page about what you do with it.

Pagination and bigger responses

APIs return large result sets in pages. Loop until there is no next page, cap the total, and cache the combined result:

Python
@st.cache_data(ttl=3600)
def get_all_items(max_pages: int = 10):
    items, page = [], 1
    while page <= max_pages:
        data = requests.get(URL, params={"page": page}, timeout=10).json()
        if not data["items"]:
            break
        items.extend(data["items"])
        page += 1
    return items

The max_pages limit is a safety net against runaway loops. For slow APIs consider fetching in a background job and showing the stored result, so the user is never waiting on a long loop.

Common mistakes

  • No timeout on requests.get.
  • Pasting an API key into the script and committing it.
  • Calling the API on every rerun because there is no cache.
  • Caching errors, or catching exceptions inside the cached function so failures are stored.
  • Showing the full exception text (which may include a URL with the key) to end users.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why should every requests.get call have a timeout?

    Without a timeout, a stuck connection blocks the script and the user sees a spinner indefinitely.
  2. 2. Where do API keys belong in a Streamlit project?

    secrets.toml and the host's secrets settings keep credentials out of source control.
  3. 3. Why wrap the API call in @st.cache_data(ttl=600)?

    The script reruns on every interaction, so caching prevents repeated calls and protects rate limits.
  4. 4. What should you do if an API key was accidentally pushed to a public repository?

    Git history keeps the old commit and bots scan public repositories. Revoke and replace the key.

Summary

  • Fetch with requests, always with a timeout and raise_for_status().
  • Cache API calls with @st.cache_data(ttl=...) to survive reruns and respect rate limits.
  • Handle failures at the call site with try/except and st.error.
  • Store keys in .streamlit/secrets.toml (git-ignored) and read them with st.secrets.
  • Ask for user-provided keys with a password input and keep them in session state only.

Next: Editing and Saving Data