Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 111 ยท Getting Started

Baselines and Dummy Models

Learn why every machine learning project needs a baseline, how DummyClassifier and DummyRegressor work, and how to beat a baseline honestly.

Suppose a model reports 95 percent accuracy. Is that good? It depends entirely on what a trivial approach would score. If 95 percent of customers never cancel, then a "model" that always answers "stays" is already 95 percent accurate and has learned nothing. A baseline is that trivial reference. It turns a bare number into a meaningful comparison.

This lesson shows how to build baselines with scikit-learn's dummy models, why they are an essential first step in every project, and how to judge whether a real model truly beats one.

What a baseline is

A baseline is the simplest sensible predictor you could deploy without any real learning. It answers one question: what score can I get with almost no effort? Your real model must beat it clearly, or it is not worth its complexity, its cost and its risk.

Baselines come in two flavors.

  • Naive baselines: ignore the features and predict a constant, such as the most common class or the average value.
  • Simple-rule baselines: a one-line business rule, such as "predict last month's value" or "flag orders above $5,000".

Why bother with a baseline?

Gives scale A score means little without a reference.
Stops false hope Exposes models that learned nothing.
Tests the pipeline Proves your split and metric work.
Justifies cost Complex models must earn their keep.
1 / 2

DummyClassifier

DummyClassifier follows the normal estimator API (fit, predict, score) but ignores the features. Its strategy argument chooses what it predicts.

Strategy Behavior
most_frequent Always predict the most common class in the training data
prior Same prediction, but predict_proba returns the class shares
stratified Random guesses that follow the training class shares
uniform Random guesses, all classes equally likely
constant Always predict a class you name
Python The accuracy trap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The most_frequent dummy reaches about 95 percent accuracy without using a single feature. A model reporting 95 percent would therefore be no better than guessing "negative" every time. This is the accuracy trap: on imbalanced data, accuracy flatters useless models. The fix is to choose better metrics, covered in classification metrics, and to always compare with the baseline.

DummyRegressor

DummyRegressor does the same for numeric targets. Its default strategy predicts the mean of the training targets. Other strategies are median, quantile and constant. Its score is R-squared, and for the mean predictor on the training data that value is exactly 0, which gives a nice interpretation: R-squared measures how much better you are than predicting the average.

Python Mean and median predictors

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both predictors return one constant for every row. Their R-squared on the test set sits near zero, and may even be slightly negative, because the test mean differs a little from the training mean. Negative R-squared simply means "worse than predicting the test average".

Beating a baseline honestly

Now the real comparison. A model needs to beat its baseline on held-out data, using the metric that matters, and by a margin that is not just luck.

Python Baseline versus a real model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at what each metric says. Accuracy rises only a little, because the baseline starts so high. Recall on the rare class tells the real story: the baseline catches none of the positives, while the model catches some. If you only quoted accuracy, you would barely notice the improvement; the right metric reveals it.

A checklist for an honest comparison

  • Use the same test split for the baseline and the model, never different data.
  • Compare on the metric tied to your goal, not the one that looks best.
  • Include a simple-rule baseline if a business rule exists. A model that cannot beat a sensible rule is not ready.
  • Check that the gain is larger than the random variation between splits. Cross-validation gives that variation.
  • Remember the cost: a tiny gain from a model that is slow, opaque or fragile may not be worth shipping.

Make "fit the dummy model" the second thing you do in every project, right after the train/test split. It takes one line, and it will save you from many misleading results.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Data has 98 percent negatives. A DummyClassifier(strategy='most_frequent') is evaluated. Roughly what accuracy does it get?

    It always predicts the majority class, so it is right whenever the true label is negative, about 98 percent of the time.
  2. 2. What does DummyRegressor predict by default?

    The default strategy is 'mean', a single constant equal to the average training target.
  3. 3. A model scores 96 percent accuracy; the majority-class baseline scores 95 percent. What is the best conclusion?

    A one-point gain over the baseline is weak evidence, and accuracy hides how the rare class is handled.
  4. 4. What does a negative R-squared on the test set mean?

    R-squared compares with the mean predictor. Below zero means the model does worse than that trivial reference.

Summary

  • A baseline is a trivial reference predictor; a real model must beat it clearly on held-out data.
  • DummyClassifier (most_frequent, prior, stratified, uniform, constant) and DummyRegressor (mean, median, quantile, constant) ignore the features.
  • On imbalanced data, a majority-class baseline has high accuracy, so accuracy alone is misleading.
  • R-squared compares a model with the mean predictor; zero means no better than the average.
  • Compare on the same split, with the right metric, and include simple-rule baselines.

Next: Linear Regression