Skip to content
Tutorials Center
My learning

Topics

Lesson 40 of 40 ยท Projects and Practice

scikit-learn Practice Exercises

Practice scikit-learn with eight exercises on cross-validation, metrics, tuning, clustering, PCA, pipelines, leakage and saving models, each with a runnable solution you can edit.

You learn machine learning by doing, so this lesson is a set of exercises. Each one states a task, then shows a runnable solution in an editor. Try to write your own code first (change the starter or delete it and write from scratch), compare it with the solution, and then experiment: change a seed, a metric, a model.

Rules of thumb for all exercises: set random_state, split or cross-validate before judging, compare against a baseline, and keep preprocessing inside a pipeline.

Exercise map

1-3 Evaluate, compare, tune.
4-5 Pipelines and leakage.
6-7 Clustering and PCA.
8 Save and reload a model.

Do them in order or jump to the topic you need.

1 / 2

Exercise 1: Beat the baseline

Task. On the breast cancer dataset, compute the cross-validated accuracy of a DummyClassifier (most frequent class) and of a scaled LogisticRegression. How much better is the real model?

Python Solution 1

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 2: Pick the right metric

Task. Create a dataset where only 3% of rows are positive. Train a random forest and report accuracy, recall and average precision on a stratified test split. Explain why accuracy looks great.

Python Solution 2

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Task. Tune the n_neighbors (1-30) and weights of a k-nearest-neighbors classifier on the wine dataset with RandomizedSearchCV inside a scaling pipeline. Report the best parameters and the test score.

Python Solution 3

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 4: Build a mixed-type pipeline

Task. Given a small DataFrame with age (with gaps), income, city and plan, build a ColumnTransformer that imputes and scales the numbers and one-hot encodes the categories, followed by logistic regression. Predict for a customer from a city never seen in training.

Python Solution 4

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 5: Find the leak

Task. A colleague selects the best 10 of 500 random features on all the data and then cross-validates, getting 80% accuracy on random labels. Reproduce the error, then fix it with a pipeline and show the honest score.

Python Solution 5

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The leaky score is far above 0.5 even though no real signal exists. The pipeline reports roughly chance level, which is the truth.

Exercise 6: Cluster and name the groups

Task. Cluster the iris measurements into 3 groups with a scaled k-means. Compare the clusters with the real species using a cross-tabulation and the adjusted Rand index.

Python Solution 6

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 7: Compress with PCA

Task. On the digits dataset, find the smallest number of principal components that keeps 90% of the variance, then show that a classifier on those components is nearly as accurate as on all 64 pixels.

Python Solution 7

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 8: Save, reload and verify

Task. Train a pipeline, save it to an in-memory joblib buffer along with the scikit-learn version and the feature names, reload it, and verify that predictions on 5 stored rows are identical.

Python Solution 8

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stretch goals

  • Re-do exercise 3 with GridSearchCV and compare how many fits each search needed.
  • In exercise 5, put StandardScaler before the selector and check that the result does not change.
  • For exercise 6, try AgglomerativeClustering and GaussianMixture and compare the ARI.
  • Pick any dataset from earlier lessons and go through the whole checklist: baseline, pipeline, cross-validation, tuning, test set, saved bundle.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which comparison tells you whether a model learned anything useful?

    A baseline gives context. Only a cross-validated improvement over it is evidence of learning.
  2. 2. Why does feature selection on all data before cross-validation inflate scores?

    Information about the held-out labels leaks into the choice of features.
  3. 3. What does PCA(n_components=0.90) choose?

    A float between 0 and 1 is read as the target explained variance ratio.
  4. 4. Why store check rows and expected predictions in a model bundle?

    A quick regression test after deployment catches version or preprocessing mismatches.

Summary

  • Always compare with a baseline and use cross-validation or a held-out test set.
  • Match the metric to the problem; accuracy misleads on rare classes.
  • Pipelines remove leakage and make mixed-type preprocessing reproducible.
  • Clustering and PCA need scaling and honest evaluation just like supervised models.
  • Save pipelines with metadata and a verification sample.

Back to the track overview: scikit-learn