Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 38 ยท Projects and Practice

Project: Machine Learning Prediction App

Build a complete regression app in Streamlit: train and evaluate a model offline, show predictions with an uncertainty range, explain inputs and package it for deployment.

In this project you will build a prediction app around a regression model. A user enters a few measurements, and the app returns a numeric prediction, a range that shows how uncertain it is, a comparison with the training data, and a short model card. The dataset is the diabetes dataset bundled with scikit-learn (442 patients, ten standardized baseline measurements, and a numeric score of disease progression one year later). It is a teaching dataset, so the app is a demonstration of the technique, not a medical tool, and it says so on the page.

You will reuse ideas from Building a Machine Learning App, Saving, Loading and Sharing Models and Model Evaluation Dashboards. The new parts are regression-specific: error metrics, a prediction interval, and a "how typical is this input" check.

Streamlit does not run on this site, so the app file is read-only. All the model code is runnable here in real scikit-learn.

Project structure

Text
diabetes-app/
โ”œโ”€โ”€ train.py             # fit, evaluate, save model + metadata
โ”œโ”€โ”€ model_utils.py       # prediction helpers (tested)
โ”œโ”€โ”€ app.py               # Streamlit UI
โ”œโ”€โ”€ requirements.txt
โ””โ”€โ”€ artifacts/
    โ”œโ”€โ”€ model.joblib
    โ””โ”€โ”€ metadata.json

Training is separate from serving, as the persistence lesson recommended. For this lesson the TryIts keep everything in memory.

Step 1: train and measure

We use a random forest. The target is a continuous number, so the metrics are mean absolute error (MAE, average miss in the target's units), root mean squared error (RMSE, punishes big misses) and R-squared (share of variance explained). Always compare against a baseline: a model that always predicts the training mean. If your model barely beats it, the features carry little signal.

Python train.py: fit and evaluate against a baseline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The forest should clearly beat the baseline, but notice that the error is still large relative to the target range. That is honest information to put on the page: this problem is hard and the model is useful only as a rough guide.

Step 2: an uncertainty range

A single number invites false precision. A random forest gives a cheap measure of spread: ask every tree for its prediction and look at the percentiles. The result is not a statistically guaranteed interval, but it shows when the trees disagree, which usually means the input is unusual or the data are noisy.

Python model_utils.py: prediction with a range

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The range from predict_with_range and the list from typicality are exactly what the page will show. A wide range or a non-empty list triggers a warning.

Step 3: calibrate the range with test errors

The tree spread is only a hint. A more honest, simple alternative: look at the model's errors on the test set and report that "about 80 percent of the time the real value was within this distance of the prediction". This quantity is computed once and shown as a plain-language sentence.

Python An empirical error band

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The sentence "Typically within ยฑN points" is easy for a reader to understand and is based on measurement, not on the model's own confidence.

Step 4: the app (read-only)

Python
import json
import joblib
import numpy as np
import pandas as pd
import streamlit as st
from sklearn.datasets import load_diabetes

st.set_page_config(page_title="Progression estimator", page_icon="๐Ÿ“Š", layout="wide")

@st.cache_resource
def load_artifacts():
    model = joblib.load("artifacts/model.joblib")
    with open("artifacts/metadata.json") as f:
        meta = json.load(f)
    return model, meta

@st.cache_data
def load_reference():
    X, y = load_diabetes(return_X_y=True, as_frame=True)
    return X, y

model, meta = load_artifacts()
X, y = load_reference()

st.title("Disease progression estimator")
st.warning("Teaching demo built on a public sample dataset. Not medical advice.")

with st.sidebar:
    st.header("Inputs (standardized scale)")
    values = {}
    for column in X.columns:
        values[column] = st.slider(column, float(X[column].min()), float(X[column].max()),
                                   float(X[column].median()), 0.005, format="%.3f")
row = pd.DataFrame([values], columns=X.columns)

per_tree = np.array([tree.predict(row.to_numpy())[0] for tree in model.estimators_])
prediction = float(per_tree.mean())
low, high = np.percentile(per_tree, [10, 90])

left, middle, right = st.columns(3)
left.metric("Estimated score", f"{prediction:.0f}")
middle.metric("Tree range (10th to 90th)", f"{low:.0f} to {high:.0f}")
right.metric("Typical test error", f"ยฑ{meta['error_band_80']:.0f}")

st.subheader("Where does this land?")
st.bar_chart(pd.cut(y, 12).value_counts().sort_index().rename(lambda i: f"{i.left:.0f}").rename("patients"))
st.caption(f"The estimate is {prediction:.0f}; the training scores run from {y.min():.0f} to {y.max():.0f}.")

lo, hi = X.quantile(0.01), X.quantile(0.99)
unusual = [c for c in X.columns if not lo[c] <= row[c].iloc[0] <= hi[c]]
if unusual:
    st.warning("Unusual inputs, the estimate is less reliable: " + ", ".join(unusual))

with st.expander("Model card"):
    st.markdown(f"""
    - **Model:** random forest, {len(model.estimators_)} trees
    - **Data:** {meta['n_train']} training patients, {meta['n_test']} held out
    - **Test MAE:** {meta['mae']:.1f}, **Rยฒ:** {meta['r2']:.2f}
    - **Limits:** small dataset, features are standardized, not suitable for decisions about care.
    """)

Notice the page leads with the warning, always shows an uncertainty figure next to the number, and has a model card. Those habits matter more than any chart.

Step 5: test and ship

  • Put predict_with_range and typicality in model_utils.py and test them with Pest-style assertions in a normal Python test (pytest): the range must contain the prediction, and an extreme input must appear in the unusual list.
  • Pin scikit-learn, pandas, numpy, joblib and streamlit in requirements.txt (Deploying to Streamlit Community Cloud).
  • Run training in CI or locally and commit the small artifacts/ files, or fetch them from storage at startup with a checksum.

Ideas to extend

  1. A second model (gradient boosting) and a selectbox to compare them, with the metrics table side by side.
  2. A "what if" panel: change one feature and plot the prediction across its whole range (st.line_chart).
  3. Batch mode: upload a CSV, predict all rows and offer a download (Files, CSV and Excel).
  4. Log every prediction (with consent) to a table and chart how inputs drift over time.

Common mistakes

  • Reporting a single number with no error information.
  • Judging the model by training error.
  • Skipping the baseline, so a weak model looks impressive.
  • Presenting a demo model as a decision tool.
  • Retraining in the app instead of loading a saved artifact.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why compare a regression model with a mean-predicting baseline?

    If the model barely beats the baseline, the features carry little information, whatever the metric looks like.
  2. 2. What does the spread of per-tree predictions in a random forest indicate?

    Large disagreement often means unusual inputs or noisy data. It is a hint, not a guaranteed interval.
  3. 3. Why also report an error band computed from the test set?

    An empirical band like 'within ยฑN points 80 percent of the time' reflects actual performance.
  4. 4. Which is the best practice for the app's relationship to training?

    Training is slow and belongs outside the request path. The app should only load and use the saved model.

Summary

  • Evaluate regression models with MAE, RMSE and R-squared against a simple baseline.
  • Show uncertainty: per-tree spread as a hint and a measured error band as a plain-language statement.
  • Flag unusual inputs by comparing them with the training data's percentile range.
  • Train offline, save artifacts and metadata, load once with cache_resource.
  • Lead with limits (a warning and a model card); a demo must not pose as a decision tool.

Next: Project: Data Explorer