Skip to content
Tutorials Center
My learning

Topics

Lesson 11 of 41 ยท Data and Experiment Tracking

Data Validation and Schemas

Stop bad data before it reaches your model: write schema checks for types, ranges and missing values, validate new batches against training statistics, and fail fast in pipelines.

Most production ML failures are data failures: a column disappears, a unit changes, a sensor sends zeros, a join duplicates rows. The model does not raise an error; it just produces nonsense. Data validation adds explicit checks at the entrance of every pipeline so bad data stops loudly instead of flowing quietly.

Think of a schema as a contract between the data producer and your model. It says which columns exist, what types and ranges they have, how many nulls are acceptable, and what the data should look like statistically.

Three levels of checks

What to validate

Structure Columns, types, uniqueness.
Values Ranges, allowed categories, nulls.
Statistics Means, rates, drift versus training.

Cheap checks first, statistical checks last.

1 / 2

A schema check from scratch

The checks below are plain pandas, so they run anywhere. Each rule returns a list of problems; an empty list means the batch is valid.

Python A hand-written schema validator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that bad triggers several problems at once. Reporting all problems in one pass saves debugging time compared with stopping at the first.

Failing fast in a pipeline

Validation is only useful if it can stop the pipeline. Raise an exception (or return a failing status in your orchestrator) when critical checks fail, and only warn for minor ones.

Python Errors versus warnings

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Statistical checks against training data

A new batch can match the schema and still be strange. Compare simple statistics against what the model saw in training. A tolerance based on the training standard deviation is a quick first alarm.

Python Compare a batch to training statistics

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Fahrenheit batch is flagged immediately. The z-score test compares the batch mean with the training mean; the full drift detection chapter later in the course covers distribution tests in more depth.

Real tools

Libraries package these ideas with richer rules, reports and integrations: Great Expectations, Pandera, TensorFlow Data Validation (TFDV), Pydantic for row-level checks, and Evidently for data quality reports. The code below shows the Pandera style (read-only, it needs the package installed in your environment):

Python
import pandera as pa

schema = pa.DataFrameSchema({
    "age": pa.Column(int, pa.Check.in_range(0, 120)),
    "country": pa.Column(str, pa.Check.isin(["DE", "US", "FR"])),
})
schema.validate(df)     # raises SchemaError on failure

Data contracts

A data contract turns the schema into an agreement with the team that produces the data: columns, types, freshness (data must arrive by 06:00), and who to call when it breaks. Contracts shift problems upstream, where they are cheapest to fix.

Generate the first schema from healthy historical data, then tighten it by hand. A schema nobody reviews drifts into uselessness.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why validate data at the entrance of a pipeline?

    Models rarely raise errors on bad data; explicit checks do.
  2. 2. A batch has the right columns but every temperature is 3 times higher than usual. Which check catches this?

    Structure is fine; only statistics reveal the shift.
  3. 3. What is a good practice when validating?

    A complete report speeds up diagnosis.
  4. 4. What is a data contract?

    It makes expectations explicit and assigns responsibility.

Summary

  • Validate structure, values and statistics before training and before scoring.
  • Fail fast on critical problems; warn on minor ones.
  • Compare new batches with profiles saved at training time.
  • Great Expectations, Pandera and TFDV package these checks; data contracts make them organizational.

Next: Feature Engineering and Feature Stores