Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 41 ยท Governance and Tooling

Lineage and Audit Trails

Record which data, code, parameters and approvals produced each model and prediction, so you can reproduce results, answer audits and trace incidents.

Imagine a customer disputes a decision made by your model three months ago. To answer, you need to know which model version made it, what data that version was trained on, which code and parameters produced it, who approved its release and what the input looked like. Lineage is the chain that links all of those. An audit trail is the tamper-evident record of what happened and who did it.

Without lineage, retraining is guesswork and incidents turn into archaeology. With it, "what changed?" becomes a lookup. This lesson shows how to record lineage cheaply, how to query it, and how to make a log hard to alter silently.

The lineage chain

flowchart LR
  D["Dataset snapshot (fingerprint)"] --> T["Training run (code commit, params)"]
  C["Code commit"] --> T
  T --> M["Model version"]
  M --> A["Approval and release"]
  A --> S["Serving deployment"]
  S --> P["Prediction (request id, model version)"]

Each arrow is a recorded link. Follow it backward from any prediction to the exact data.

Minimum links to record:

Link Stored with Example value
Data training run content hash of the snapshot
Code training run git commit id
Parameters training run the full config
Environment training run lockfile hash, library versions
Model registry version number, metrics
Approval registry approver, time, gate results
Prediction request log request id, model version, timestamp

Capture lineage in a run record

Python A run record with automatic lineage

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two hashes are the key idea: a data fingerprint proves which data was used, and a model hash proves which artifact was produced. Experiment trackers and registries (MLflow, DVC, Weights and Biases and others) store these automatically; the principle is the same.

Answer a question by walking the graph

Store records in any database and link them by id. Answering "which data trained the model that made this prediction?" is then three lookups.

Python Trace a prediction back to its data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because every prediction logged its model version (see Building a Prediction Service), the answer takes milliseconds, not days.

Make the audit log tamper-evident

A log anyone can edit does not prove anything. A simple technique is a hash chain: each entry includes the hash of the previous entry, so changing an old entry breaks every later hash.

Python A hash-chained audit log

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In production, write audit records to append-only storage with restricted permissions, and ship copies to a separate system. A hash chain adds evidence of tampering; it does not prevent deletion, so access control still matters.

Privacy and retention

Lineage and logs can contain personal data. Decide up front:

  • What to log: ids and hashes rather than raw inputs when possible.
  • How long to keep it: match legal requirements, then delete.
  • Who may read it: audit logs usually need tighter access than application logs.
  • Deletion requests: know which datasets and models contain a person's data so you can act on them.

Common mistakes

  • Recording the name of a dataset (customers.csv) instead of its content hash, so you cannot tell which version was used.
  • Logging predictions without the model version.
  • Keeping lineage in people's notebooks or chat messages.
  • Letting the pipeline and the log live in the same writable place.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a data fingerprint prove?

    A content hash changes whenever the data does, so it pins the exact version.
  2. 2. Which field must every logged prediction include for lineage to work?

    The model version links a prediction to its run, data and approval.
  3. 3. What does a hash chain give an audit log?

    Changing an old entry breaks the hashes of all later entries, which verification detects.
  4. 4. Why record a dataset's hash instead of only its file name?

    The name does not change when the data inside it does.

Summary

  • Lineage links prediction, model, run, code, parameters, environment and data.
  • Record content hashes and versions, not just names.
  • Log the model version with every prediction.
  • Hash chains and append-only storage make audit logs tamper-evident.
  • Plan privacy, retention and access for lineage data.

Next: The MLOps Tooling Landscape