Skip to content
Tutorials Center
My learning

Topics

Lesson 78 of 111 ยท Beyond the Basics

Serving Models with Flask and Render

Wrap a scikit-learn model in a Flask API with validation, test it, package it with gunicorn and requirements, and deploy it to a cloud host such as Render.

A model that lives in a notebook helps nobody except you. To let an app, a website or another team use it, you put it behind a web API: a program that listens for HTTP requests, loads the model once, and answers each request with a prediction. Saving models and deploying showed the principles and a FastAPI service. This lesson builds the same kind of service with Flask, a small and very common Python web framework, and then walks through putting it on the internet with a hosting service such as Render.

What a model service needs

  1. Load once, predict many times. Read the saved model when the process starts, not on every request.
  2. A clear contract. Say which JSON fields are accepted, in which units, and what comes back.
  3. Validation. Reject missing, non-numeric and absurd values with a helpful error before they reach the model.
  4. A health endpoint. Hosting platforms call it to check that the app is alive.
  5. A model version in every response, so you can tell which model produced a prediction.
  6. A production server. Flask's built-in server is for development only. Use gunicorn or a similar WSGI server.

A complete Flask service you can run here

The example trains a small classifier on the iris flowers, builds a Flask app with a health check and a prediction endpoint, and then tests the app with Flask's test client. The test client calls your routes directly, without opening a network port, so it also works inside the browser. Flask is installed with micropip (the first run downloads it).

Python A Flask prediction API with validation, tested in-process

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Every bad request gets a specific status code (400 for a malformed request, 422 for a well-formed request with unacceptable values) and a message that says what to fix, and the model is never called with data it was not designed for. Good requests return the label, the probability and the model version. Writing such tests with the test client is the cheapest insurance you can buy, and they belong in your CI pipeline.

The files you need on your own machine

The code above is a single file in a real project. A minimal repository looks like this:

TEXT
iris-service/
  app.py              # the Flask app (the code above, loading model.joblib at start-up)
  train.py            # trains the model and writes model.joblib
  model.joblib        # the saved model (or download it at start-up from storage)
  requirements.txt    # pinned dependencies
  tests/test_api.py   # the test-client checks

In app.py, replace the training lines with loading the saved file:

Python
# read-only: runs in your own project, not in the browser
import joblib
model = joblib.load("model.joblib")

Pin the exact library versions in requirements.txt. A model saved with one scikit-learn version may not load with another, which is the training/serving skew problem again:

TEXT
flask==3.1.*
gunicorn==23.*
scikit-learn==1.8.*
numpy==2.*
joblib==1.5.*

Running it with gunicorn

Flask's development server handles one request at a time and is not hardened. In production a WSGI server such as gunicorn starts several worker processes, each with its own copy of the model in memory:

Terminal
# read-only: run these in a terminal on your own machine
pip install -r requirements.txt
gunicorn --workers 2 --bind 0.0.0.0:8000 app:app
curl -X POST localhost:8000/predict -H "Content-Type: application/json" \
     -d '{"sepal_length": 5.1, "sepal_width": 3.5, "petal_length": 1.4, "petal_width": 0.2}'

app:app means "the variable app in the file app.py". Choose the number of workers from your CPU cores and your memory: a model that takes 500 MB multiplied by four workers needs 2 GB.

Deploying on Render

Render is one of several platforms (others: Railway, Fly.io, Google Cloud Run, AWS, Heroku-style services) that builds your repository and keeps it running on a public URL. The details of their dashboards change, so check the current documentation, but the steps follow the same pattern:

  1. Push the project to a Git repository (GitHub or GitLab). Keep large data out of it; the small model file is fine, or store it in object storage and download it at start-up.
  2. Create a new Web Service in the Render dashboard and connect the repository.
  3. Fill in the build and start commands. Build command: pip install -r requirements.txt. Start command: gunicorn app:app. Choose a Python version that matches your local one (for example through an environment variable or a .python-version file; see Render's current docs).
  4. Set environment variables for configuration and secrets (API keys, the model path). Never commit secrets to Git.
  5. Set the health check path to /health, so the platform restarts the service if it stops answering.
  6. Deploy. Every push to the connected branch triggers a new build. Render shows the logs and the public URL.

Many platforms can instead build from a Dockerfile, which guarantees the same environment everywhere:

DOCKERFILE
# read-only: a minimal container for the service
FROM python:3.13-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["gunicorn", "--workers", "2", "--bind", "0.0.0.0:8000", "app:app"]

Many free tiers put an idle service to sleep, so the first request after a pause is slow (a cold start, during which the model is loaded again). The disk of a running service is usually temporary as well: anything written at run time can disappear on the next deploy, so write logs to standard output and keep durable data in a database or object store.

Production checklist

  • Validate every input, and log rejected requests (without logging personal data).
  • Add a timeout and a limit on request size.
  • Return the model version and keep it in your logs.
  • Protect the endpoint if it is not public: an API key or token, HTTPS (platforms provide it), rate limiting.
  • Monitor latency, errors and input drift (monitoring).
  • Roll out new versions gradually and keep the old one ready for a rollback.
  • Load-test before launch: ask how many requests per second one worker can handle with your model.

Flask or FastAPI?

Both work. Flask is minimal and everywhere, and you write validation yourself. FastAPI, shown in the deployment lesson, validates inputs with type hints, generates interactive documentation and has first-class async support. If you only need a quick demo for people, Streamlit and Gradio are even faster.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why should the model be loaded when the app starts and not inside the request handler?

    Read the model into memory once. Each request then only runs the fast predict call.
  2. 2. What is the difference between a 400 and a 422 response in the example?

    Status codes tell clients what to fix: structure (400) or values (422).
  3. 3. Why use gunicorn instead of Flask's built-in server in production?

    The development server is single-purpose and not hardened. A WSGI server manages workers, timeouts and restarts.
  4. 4. What is a cold start on a platform that sleeps idle services?

    The platform has to start the process again, and the app must reload the model before it can answer.

Summary

  • A model service loads the model once, validates inputs, exposes a health check and reports its model version.
  • Flask's test client lets you test the API without a network, even in the browser.
  • Pin dependency versions, serve with gunicorn, and keep secrets in environment variables.
  • A platform such as Render builds from your repository with a build command and a start command and restarts the service by calling the health path.
  • Expect cold starts and temporary disks on small plans, and monitor the service after launch.

Next: keep track of experiments and models with Experiment Tracking with MLflow.