Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 41 ยท Governance and Tooling

Fairness and Bias Checks

Measure group-level differences in selection rate and error rates, understand the trade-offs between fairness definitions, and add bias checks to the release gate.

A model can be accurate on average and still treat groups of people very differently. If it helps decide who gets a loan, an interview or extra medical attention, those differences matter ethically and, in many places, legally. MLOps is where fairness becomes routine: measured at every release, tracked in production and recorded in the model card.

This lesson introduces the most common group metrics, shows how to compute them, explains why they cannot all be satisfied together, and turns them into an automatic check. It is an engineering introduction, not legal advice, and no metric replaces human judgment about harms.

Where bias comes from

Sources of bias

Historical data Past decisions were unfair.
Missing groups Some people are rare in the data.
Label noise Labels measure a flawed proxy.
Feedback loops The model shapes its future data.

Bias enters long before the algorithm.

1 / 2

Three group metrics

Let A be a group attribute (for example, region or age band, only where it is lawful and appropriate to use it for auditing) and Y the true outcome. Compare groups on:

  • Selection rate (demographic parity): the share of each group receiving the positive prediction. Equal rates mean demographic parity.
  • True positive rate (equal opportunity): among people who truly deserve the positive outcome, the share the model finds. Equal rates mean equal opportunity.
  • False positive rate: among those who do not, the share wrongly flagged. Equal TPR and FPR together is equalized odds.

A common summary is the ratio or difference between groups. Selection-rate ratios are sometimes compared with the "four-fifths" rule of thumb from employment practice (a ratio below 0.8 deserves a look), but it is a screening hint, not a legal verdict.

Python Group metrics from predictions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Even though both groups have the same true rate of need, group B has a lower TPR (more truly needy people missed) and a higher FPR (more wrongly flagged). The cause here is a noisier signal for B, which is typical when a group is under-represented in the training data.

You cannot have everything

When the real-world base rates differ between groups, a model that is not perfect cannot satisfy all fairness definitions at once. Equal selection rates, equal error rates and equal precision conflict. So the work is not to find "the" fair metric but to:

  1. Choose the metric that matches the harm you are worried about. If missing a qualified person is the harm, watch TPR. If wrongly flagging is the harm, watch FPR.
  2. Make the choice explicitly, with stakeholders, and write it in the model card.
  3. Monitor it over time, because the gaps move when the data does.

Add a bias check to the gate

Python A fairness gate with sample-size protection

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The gate refuses to judge groups that are too small (its own kind of warning) and reports the size of each gap. Treat a failure as a prompt to investigate, not an automatic verdict.

What to do when a check fails

  • Look at the data first: is a group underrepresented, or are its labels worse?
  • Collect more data for weak groups where possible.
  • Reweight or resample during training, or use fairness-aware libraries (Fairlearn and AIF360 are well-known Python options).
  • Adjust thresholds per group only where law and policy allow it, and document it.
  • Add human review for borderline cases.
  • Reconsider the use: sometimes the right answer is not to automate this decision.

Common mistakes

  • "Fairness through unawareness": dropping the sensitive column does not remove bias, because other features act as proxies. You also need the attribute (or a lawful stand-in) to measure gaps.
  • Computing metrics on groups with a handful of rows.
  • Checking once at launch instead of monitoring in production.
  • Treating a single metric as proof the system is fair.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Equal opportunity asks that groups have equal:

    Among people who truly qualify, each group should be found at the same rate.
  2. 2. Why can't all fairness metrics be satisfied at once in general?

    Known impossibility results show equal selection, error rates and precision conflict unless the model is perfect or base rates are equal.
  3. 3. Why is removing the sensitive column not enough?

    Unawareness leaves proxy leakage in place and removes your ability to audit.
  4. 4. What should a fairness gate do with a group of 12 rows?

    Small samples give unstable metrics; surface the limitation rather than guessing.

Summary

  • Bias can enter through data, labels, modeling choices and feedback loops.
  • Compare groups on selection rate, TPR and FPR, with sample sizes.
  • Definitions conflict; pick the metric that matches the harm and document the choice.
  • Put checks in the gate and keep monitoring after release.
  • Fixes include better data, reweighting, thresholds (where allowed), human review or not automating.

Next: Lineage and Audit Trails