Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 41 ยท ML Lifecycle and Pipelines

A Reproducible Training Script

Turn a notebook into a reproducible training script: fix seeds, split data deterministically, pin settings, save a run record, and prove two runs give identical results.

A result you cannot reproduce is an opinion. Reproducibility means that someone else (or you, next month) can re-run the same code on the same data with the same settings and get the same model. It is the foundation of everything else in MLOps: debugging, audits, rollbacks and fair comparisons all depend on it.

Notebooks make this hard, because cells run in any order and keep hidden state. A training script with explicit inputs and outputs removes the ambiguity. This lesson shows the ingredients.

What can change between two runs

Sources of drift between runs

Randomness Seeds, shuffling, init.
Data Rows added or reordered.
Libraries New versions change results.
Settings Parameters edited by hand.

Pin all four to reproduce a run.

1 / 2

Step 1: seed every random source

Many algorithms use randomness: shuffling, bootstrap samples, weight initialization, train/test splits. Pass an explicit random_state everywhere and keep the seed in one place.

Python Seeds make runs identical

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Runs with the same seed match. Different seeds differ, which is normal: report results over several seeds when you compare models, not one lucky number.

Step 2: split deterministically

Never split by calling a random function you cannot repeat. Use a fixed seed, or better, split by a stable key (such as a hash of an ID) so a row always lands in the same set even when new rows arrive.

Python A split that survives new data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A plain train_test_split on the grown table would reshuffle everyone and leak yesterday's test rows into today's training set. Hash-based splitting avoids that.

Step 3: put settings in one place

Collect every tunable value in a single dictionary (or config file, see configuration and parameters). The training function takes it as input, so the run record can store it.

Step 4: write a run record

A run record is the receipt for a training run. At minimum store the parameters, the metrics, the data fingerprint, the code version and the library versions.

Python A complete reproducible script

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two records are equal, so the run is reproducible on this setup. If a colleague gets a different AUC, the version block and the data fingerprint tell you where to look first.

Step 5: make it a command

Move the notebook logic into a file with a main() that reads a config and writes outputs. Keep the notebook for exploration only.

Python
# train.py (read-only example)
import argparse, json

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--config", default="config.json")
    parser.add_argument("--out", default="artifacts/")
    args = parser.parse_args()
    cfg = json.load(open(args.config))
    # ... train, evaluate, save model + run record into args.out ...

if __name__ == "__main__":
    main()

Run it as python train.py --config config.json. A CI job or scheduler can run the exact same command.

Pin the environment

Lock library versions in a requirements.txt or lock file (pip freeze, uv lock, poetry.lock). The packaging lesson goes deeper. Also note that floating-point results can differ slightly across CPUs and library builds, so compare metrics with a tolerance, not exact equality, across machines.

Setting a seed makes results repeatable, not correct. A lucky seed is still a lucky seed.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which is NOT a common source of differences between two training runs?

    Randomness, data, libraries and settings change results; an editor theme does not.
  2. 2. Why split data by a hash of an ID rather than by a random shuffle?

    A stable key gives a stable assignment, so test rows never move into training.
  3. 3. What belongs in a run record?

    The record lets you understand and reproduce the run later.
  4. 4. Why compare metrics across machines with a tolerance?

    Different CPUs and builds can change low-order digits.

Summary

  • Reproducibility means same code, data, settings and environment give the same model.
  • Seed all randomness and report results over several seeds when comparing.
  • Split by a stable key so rows keep their set.
  • Keep settings in one config and write a run record for every run.
  • Turn notebooks into scripts with explicit inputs and outputs.

Next: scikit-learn Pipelines for Production