Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 38 ยท Machine Learning Apps

Model Evaluation Dashboards

Build a model evaluation page in Streamlit: accuracy, precision, recall, confusion matrix, feature importance and an interactive decision-threshold slider.

A prediction form shows what a model says. An evaluation dashboard shows whether you should believe it. Stakeholders ask: how often is it right, what kinds of mistakes does it make, and what happens if we change the cutoff? Streamlit is a very good tool for answering those questions interactively, because moving a slider can recompute precision and recall instantly.

This lesson builds the numbers first (runnable scikit-learn) and then shows the Streamlit code that displays them. The decision-threshold slider is the centerpiece: it teaches the precision/recall trade-off better than any table.

Streamlit does not run on this site, so app code is read-only. All metric calculations below run for real in your browser.

Split data and evaluate on held-out rows

Evaluation only means something on data the model did not train on. Always hold out a test set, and keep the split fixed with random_state so the dashboard does not change between reruns:

Python
@st.cache_data
def get_split():
    X, y = load_breast_cancer(return_X_y=True, as_frame=True)
    return train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)

The stratify=y argument keeps the class proportions equal in both parts. In the TryIt below the same split feeds every metric.

The core metrics

For a classifier you usually need more than accuracy:

Metric Question it answers
Accuracy What share of predictions were right? Misleading when classes are unbalanced
Precision Of everything predicted positive, how much really was?
Recall Of all real positives, how many did we catch?
F1 A single balance of precision and recall
Confusion matrix Exactly which errors: false positives versus false negatives
Python Compute the evaluation numbers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a Streamlit page those numbers become a row of cards:

Python
cols = st.columns(4)
for col, (name, value) in zip(cols, metrics.items()):
    col.metric(name.title(), f"{value:.1%}")

st.subheader("Confusion matrix")
st.dataframe(
    pd.DataFrame(cm, index=["Actually 0", "Actually 1"], columns=["Predicted 0", "Predicted 1"])
)

st.metric also takes a delta= argument, handy for comparing with a previous model: col.metric("F1", f"{f1:.1%}", delta=f"{(f1 - old_f1):+.1%}").

The threshold slider

Most classifiers output a probability, and predict simply uses 0.5 as the cutoff. Moving the cutoff trades one kind of error for another:

  • A lower threshold flags more cases: recall goes up, precision goes down.
  • A higher threshold flags fewer: precision goes up, recall goes down.

Which is right depends on the cost of each mistake, which is a business decision. A medical screening test usually prefers high recall; a spam filter that deletes mail prefers high precision. Let users explore it:

Python
threshold = st.slider("Decision threshold", 0.05, 0.95, 0.50, 0.05)
proba = model.predict_proba(X_test)[:, 1]
pred = (proba >= threshold).astype(int)

Since only the cheap comparison changes, this reruns instantly, as long as the model and proba come from cached functions. Now compute the metrics across several thresholds, which is exactly the table a dashboard would plot.

Python Precision and recall at different thresholds

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

As the threshold climbs, fewer cases are flagged and recall drops. In the app, st.line_chart(table.set_index("threshold")[["precision", "recall"]]) draws both curves together.

Feature importance

Users want to know why. For linear models the coefficients show direction and strength (on standardized features), and tree models expose feature_importances_. Sort them and show the top ones:

Python
importances = pd.Series(model.feature_importances_, index=X.columns).sort_values(ascending=False)
st.bar_chart(importances.head(10))

Treat importances as hints, not explanations: correlated features split credit between them, and importance says nothing about causes. A stronger check is permutation importance (sklearn.inspection.permutation_importance), which measures how much the score drops when a feature is shuffled.

Python Permutation importance on a random forest

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Permutation importance runs the model many times, so wrap it in @st.cache_data and compute it once, not on each rerun.

Cross-validation for honest uncertainty

One test split gives one number. Cross-validation repeats the split and shows the spread, which tells users how much to trust a small difference between models:

Python
from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X, y, cv=5, scoring="f1")
st.write(f"F1: {scores.mean():.3f} ยฑ {scores.std():.3f}")
st.bar_chart(pd.Series(scores, name="F1 per fold"))

Comparing two models whose scores differ by less than the fold-to-fold spread is not evidence that one is better.

Layout for a clear page

A common structure is tabs for the different audiences, with the headline numbers on top:

Python
tab_overview, tab_errors, tab_features = st.tabs(["Overview", "Errors", "Features"])
with tab_overview:
    ...   # metric cards, threshold slider
with tab_errors:
    ...   # confusion matrix, list of misclassified rows
with tab_features:
    ...   # importances

Showing the misclassified rows is among the most useful panels: X_test[pred != y_test] lets people look at real mistakes and spot patterns, such as a particular measurement range that the model handles badly.

Common mistakes

  • Reporting accuracy alone on unbalanced data.
  • Evaluating on the training set.
  • Recomputing metrics or permutation importance on every rerun instead of caching.
  • Forgetting a fixed random_state, so numbers change when the page reloads.
  • Reading feature importance as proof of cause.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A rare-disease test labels everyone healthy and gets 99 percent accuracy. Which metric reveals the problem?

    Recall shows it caught none of the sick patients. On unbalanced data accuracy alone hides this.
  2. 2. What happens to precision and recall when you raise the decision threshold?

    A higher cutoff flags fewer cases, which are more likely correct (higher precision) but misses more real positives (lower recall).
  3. 3. Why evaluate on a held-out test set?

    Scores on training data are optimistic because the model has memorized part of it.
  4. 4. Why cache expensive evaluation computations?

    Each widget change reruns the script, so permutation importance or cross-validation would repeat unless cached.

Summary

  • Evaluate on held-out data with a fixed split; report precision, recall, F1 and the confusion matrix, not only accuracy.
  • A threshold slider makes the precision/recall trade-off visible and lets domain experts choose.
  • Use feature importances (preferably permutation importance) as hints, never as proof of cause.
  • Cross-validation shows how stable a score is.
  • Cache heavy computations and organize results in tabs, including a view of misclassified rows.

Next: Chat and LLM Apps