Skip to content
Tutorials Center
My learning

Topics

Lesson 77 of 111 ยท Beyond the Basics

Saving Models and Deploying

Save a trained model with joblib, understand pickle's security risks, avoid training/serving skew, serve predictions from a function, and monitor drift with PSI and KS.

A model that lives only in a notebook helps nobody. Sooner or later you need to save it, load it somewhere else (a web service, a nightly job, a colleague's laptop) and keep it healthy while the world changes underneath it. This step is called deployment, and most of its problems are not about machine learning at all: they are about files, versions, input checks and monitoring.

This lesson covers the practical core: how scikit-learn models are saved, why loading an untrusted file is dangerous, how to wrap a model in a function that a web API can call, how training/serving skew silently ruins predictions, and how to detect data drift with two small statistics you write yourself. The browser cannot run a real server, so the web-API code is shown read-only; everything else runs live.

The life of a deployed model

Save One file with the whole pipeline.
Serve A function or API validates and predicts.
Monitor Watch inputs and outcomes for drift.
Retrain Replace the model on a schedule or a signal.
1 / 2

Saving a model: pickle and joblib

Python's standard way to turn an object into bytes is pickle. scikit-learn recommends joblib.dump and joblib.load, which use pickle underneath but handle large NumPy arrays efficiently. A fitted estimator is just an object holding learned numbers (such as coef_), so saving it saves those numbers and the settings.

Save the whole pipeline, not just the final model. The scaler, encoder and model belong together (see Pipelines). In the demo below we write to an in-memory buffer instead of a file, because the browser has no real disk; with a real file you would write joblib.dump(model, "model.joblib").

Python Save and reload a pipeline with joblib

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A reloaded model must give exactly the same predictions as the original. Checking this on a handful of saved example rows is a cheap test to run before every release.

The security risk

Unpickling is not just reading data: the file can contain instructions that Python executes while loading. Here is a harmless demonstration. The class tells pickle "to rebuild me, call print"; a malicious file would say "call os.system" instead.

Python Why loading a pickle is running code

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Never load a pickle or joblib file from a source you do not fully trust. Treat it like running a program someone sent you. Only load files you created yourself or fetched from storage you control; consider a signed or checksummed artifact store, and never accept uploaded model files from users. If you must exchange models with untrusted parties, use a data-only format such as ONNX or store the parameters themselves (for a linear model, the coefficients and intercept in JSON).

Versioning

A pickled model also remembers which classes it needs, so it generally only loads safely with the same version of scikit-learn that saved it. Upgrading the library may break loading or, worse, change behavior silently (scikit-learn warns if it notices a mismatch). Good practice:

  • Pin exact library versions in requirements.txt and use the same ones for training and serving.
  • Save the model together with metadata: library versions, training date, data description, metrics and a version number.
  • Keep the training code and a fixed random seed so you can rebuild the model from scratch if the old file stops loading.
Python A model file with metadata

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Serving: wrap the model in a function

Deployment means someone else's program calls your model. The cleanest design is a single function that takes raw input (for example, a dictionary from a JSON request), validates it, builds the feature array in the same column order as in training, predicts and returns plain Python values. A web framework then only has to call that function.

Validation matters because real inputs are messy: missing fields, strings where numbers belong, values far outside anything seen in training. Fail early with a clear message instead of returning a confident nonsense answer.

Python A prediction function with input checks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In production the same function sits behind an HTTP endpoint. This is a typical sketch with FastAPI. It is read-only here, because the browser cannot open a network port:

Python
# read-only sketch: requires FastAPI, which the browser runner does not have
import joblib
from fastapi import FastAPI
from pydantic import BaseModel

pipeline = joblib.load("model.joblib")      # load ONCE at start-up, not per request
app = FastAPI()

class Customer(BaseModel):
    age: int
    income: float
    plan: str

@app.post("/predict")
def predict(customer: Customer):
    return predict_one(customer.model_dump())

Three habits matter more than the framework: load the model once when the process starts, log every request and prediction (you need them for monitoring), and keep the model behind a version number so you can roll back.

Serving a model with FastAPI

The sketch above showed the shape of an API. FastAPI deserves a closer look, because it is the most common way to put a scikit-learn model behind HTTP, and it gives you three things for free:

  • Request validation. You describe the input as a pydantic model with types and limits (age: int = Field(ge=18, le=100)). FastAPI checks every request against it and answers invalid ones with status 422 and a precise message, before your model sees them.
  • A typed response. response_model= makes sure the API returns exactly the documented fields.
  • Automatic documentation. FastAPI builds an OpenAPI description of your endpoints and a browser page (/docs) where you can try them.

A model service usually has at least these endpoints: /health (is the process alive and which model version is loaded), /predict (one record) and /predict-batch (a list, which is much faster than many single calls because the pipeline works on a whole table at once).

The example below is a complete, runnable version. FastAPI is installed with micropip (the first run downloads it). The browser cannot open a network port or start threads, so FastAPI's usual TestClient is not available. Instead, a small helper calls the app through the ASGI interface that FastAPI is built on, which is all a test client does. The endpoints are written with async def, which keeps them on the main thread. In your own project you would write plain def endpoints if the model call is slow (FastAPI then runs them in a thread pool), and test with from fastapi.testclient import TestClient.

Python A FastAPI model service: validation, endpoints and tests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the output as a test report. The valid request returns a prediction and a probability. The three bad requests never reach the model: each is rejected with status 422, and the message names the field that failed (['body', 'plan'], ['body', 'age'], ['body', 'income']), which is the information a client developer needs. The batch endpoint scores both customers with one call to predict_proba. The last line shows that FastAPI generated an OpenAPI description of the service on its own, including the Customer and Prediction schemas.

To run the same app for real, save it as main.py, store the trained pipeline with joblib and load it once at the top of the file, then start the server with uvicorn main:app. Then open /docs in a browser. Some habits are worth adding early:

  • Pin the environment. The scikit-learn version at serving time must match training (see the check earlier in this lesson).
  • Validate more than types. Ranges and allowed values (as in the example) catch many silent errors. Consider a "reject or warn" rule for inputs far outside the training range.
  • Return the model version in responses or headers, so you can trace any prediction back to the model that made it.
  • Log requests and predictions for monitoring, and write tests like the ones above in your CI pipeline.

Training/serving skew

Training/serving skew means the data a model sees in production is prepared differently from the data it was trained on. The model is fine; the inputs are wrong. Typical causes: a feature scaled in training but not in serving, categories encoded in a different order, a column computed with different code in two places, or a unit change (dollars versus cents).

Skew does not raise an error. The model happily returns numbers. The next example makes the failure visible by serving the same row two ways.

Python Skew: scaling forgotten at serving time

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The defenses are boring and effective:

  • One code path. Use a single Pipeline for training and serving instead of duplicating preprocessing.
  • Golden examples. Keep a small file of inputs with the expected outputs and run it against every release.
  • Log serving features and compare their distributions with the training data (next section).

Monitoring drift

Even a perfect deployment decays, because the world changes: customers behave differently, a sensor is replaced, a new marketing campaign brings a different crowd. Data drift is a change in the distribution of the model's inputs. (A change in the relationship between inputs and the true outcome is concept drift, which you can only measure once true labels arrive.)

Two simple, popular drift checks compare a feature's training distribution with its recent production values:

  • PSI, the Population Stability Index. Cut the training values into bins (often 10 quantile bins), compute the share of rows in each bin for both datasets, and sum (p_new - p_old) * ln(p_new / p_old). Common rules of thumb: below 0.1 is stable, 0.1 to 0.25 is a moderate shift, above 0.25 is a large one.
  • The Kolmogorov-Smirnov (KS) statistic: the largest gap between the two empirical cumulative distribution curves. It runs from 0 (identical) to 1 (completely separate). We compute it by hand and check it against scipy.stats.ks_2samp.
Python PSI and KS by hand

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The hand-written KS matches SciPy's, which is the check you should always run on code you write yourself. Also notice that the stable scenario still gets a small, nonzero PSI: random sampling alone creates tiny differences, so thresholds, not zero, are what you compare against.

Practical monitoring advice:

  • Run the check per feature, on a schedule (daily or weekly), and keep the history so a slow creep is visible.
  • Monitor predictions too: if the share of predicted positives jumps from 5% to 20%, investigate even before labels arrive.
  • Monitor outcomes when labels become available (days or months later) with the metrics you chose in Classification Metrics or Regression Metrics.
  • Drift is a signal to investigate, not an automatic reason to retrain. A new product launch may explain it; a broken upstream pipeline needs a fix instead.
  • Decide in advance what happens on an alert: who looks, and when a model is rolled back or retrained.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why is loading a pickle or joblib file from an unknown source dangerous?

    Unpickling rebuilds objects by calling functions named in the file. A malicious file can name any function, so only load files you trust completely.
  2. 2. What should you save so the model behaves the same at serving time as in training?

    The preprocessing steps are part of the model. Saving the full Pipeline keeps training and serving on one code path, and the metadata records which library versions can load it.
  3. 3. A feature was standardized in training, but the serving code sends raw values to the model. What is this called?

    Skew means the production inputs are prepared differently from the training inputs. It usually raises no error, so it needs tests such as golden examples.
  4. 4. A feature's PSI between training and last week's data is 0.4. How should you read it?

    By the usual rule of thumb a PSI above 0.25 is a large shift. It is a signal to investigate, which may lead to a fix, a rollback or retraining.
  5. 5. What does the KS statistic measure?

    KS compares the empirical cumulative distributions of two samples and reports the biggest vertical distance, from 0 (identical) to 1 (completely separate).

Summary

  • Save the whole pipeline with joblib.dump, together with metadata (library versions, training date, metrics), and test that a reloaded model reproduces its predictions.
  • Pickle and joblib files execute code on load: never load files from untrusted sources.
  • Pin library versions; a model file generally loads reliably only with the version that wrote it.
  • Serve through a function that validates input, fixes the column order and returns plain values; a web framework then wraps it.
  • Training/serving skew produces no errors, only wrong predictions: use one pipeline and golden examples.
  • Monitor drift per feature with PSI and KS, watch prediction rates and outcomes, and decide in advance what each alert triggers.

Next up: look at who a model helps and who it harms in Fairness, Bias and Responsible ML.