Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 37 ยท Models and Demos

Wrapping a Model in a Gradio Demo

Learn the load-once, predict-many pattern for putting any model behind a Gradio interface: global loading, preprocessing, postprocessing, labels and confidence scores.

Gradio exists to put a model in front of people. The model can be a scikit-learn classifier, a PyTorch network, a call to a hosted API or a plain Python function: Gradio only cares that you can turn inputs into outputs. This lesson shows the shape almost every model demo has, so you can apply it to any framework.

The real gradio package cannot run on this site, because it needs a Python web server and a browser front end. The Gradio code below is real and read-only; the runnable blocks use Pyodide and a tiny hand-written classifier as a labelled stand-in, so you can test the same logic that sits inside predict.

The load-once, predict-many pattern

Loading a model is slow; predicting is fast. So load the model once, when the script starts, and call it from your function for every request:

Python
import gradio as gr
from transformers import pipeline

classifier = pipeline("sentiment-analysis")   # loaded once, at import time

def predict(text):
    result = classifier(text)[0]
    return {result["label"]: float(result["score"])}

demo = gr.Interface(
    fn=predict,
    inputs=gr.Textbox(lines=3, label="Review"),
    outputs=gr.Label(num_top_classes=2),
)

demo.launch()

Starting the script prints a local URL such as http://127.0.0.1:7860. Opening it shows a text box and a Submit button; after you submit a review, a label component lists the class names with confidence bars. Because classifier lives at module level, the model is read from disk once, not on every click. Never construct the model inside predict.

Module-level state is shared by all visitors. That is fine for a read-only model, but a list or counter you modify inside predict is visible to everyone. Use state objects for per-user data.

Preprocess, predict, postprocess

A good predict has three small steps. Keeping them separate makes the function easy to test:

flowchart LR
    A["Raw input<br/>(text, image, number)"] --> B["Preprocess<br/>(clean, resize, tokenize)"]
    B --> C["Model call"]
    C --> D["Postprocess<br/>(labels, scores, formatting)"]
    D --> E["Gradio output component"]

The model call is the middle step; most bugs hide in the other two.

Gradio already does a first layer of preprocessing for you: a Textbox hands your function a str, a Number a float, an Image a NumPy array (or a PIL image or file path, depending on type=). Your own preprocessing covers what the model needs on top of that, such as lowercasing text or scaling pixel values.

Returning labels and confidences

gr.Label expects a dictionary that maps class names to probabilities. It sorts them, shows the top class big and draws bars for the rest. This is why model demos usually return {label: score} instead of a single string:

Python
def predict(text):
    probs = model.predict_proba([text])[0]
    return {name: float(p) for name, p in zip(class_names, probs)}

Convert NumPy floats to Python float; Gradio serializes the dictionary to JSON, and numpy.float32 values can fail to serialize.

A runnable stand-in: a bag-of-words classifier

This classifier counts positive and negative words. It is a stand-in for a real model, but its preprocess, predict and postprocess steps are exactly the structure you would give a real one.

Python Preprocess, predict, postprocess

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first review scores high on positive, the second low, and the third is a neutral 0.5 because no known words appear. In a real demo you would replace only model.

Handling empty and bad input

Users submit empty boxes. Decide what should happen before the model runs:

Python Guard clauses before the model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a real app you can also raise gr.Error("message"), which shows a red toast instead of returning text; see errors and performance.

Device, batching and examples

  • Device: for deep-learning models, move the model to the GPU once at load time (model.to("cuda") if available) and keep tensors on the same device in predict.
  • Batching: gr.Interface(fn, ..., batch=True, max_batch_size=8) lets Gradio group simultaneous requests into one call; your function then receives lists and returns lists. Worth it for GPU models.
  • Examples: examples=[["I love this"], ["Terrible"]] adds clickable sample inputs. Visitors who don't have an input ready will try these first.

The next lessons apply the pattern to images and audio and text.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Where should a heavy model be loaded in a Gradio script?

    Loading is slow, so do it once at startup and reuse the object in every call.
  2. 2. What does gr.Label expect from the function?

    gr.Label renders a {class: probability} mapping as a ranked list with bars.
  3. 3. Why convert numpy floats to Python floats in the returned dictionary?

    JSON encoding of numpy scalar types can raise errors, so convert with float().
  4. 4. Why split predict into preprocess, model and postprocess steps?

    Small steps are easy to test and swap; usually only the model call changes between projects.

Summary

  • Load the model once at module level; call it from a small predict function.
  • Structure predict as preprocess, model call, postprocess.
  • Return {label: probability} dictionaries for gr.Label, with plain Python floats.
  • Guard against empty or oversized input before the model runs.
  • Use batching and examples to make demos faster and friendlier.

Next: Image and Audio Demos