Lesson 107 of 111 ยท Practice and Review
Exercises: Workflow, Regression and Classification
Ten graded scikit-learn exercises with full solutions: splits, baselines, linear regression, metrics, imbalanced classification, thresholds and feature scaling.
Machine learning is learned by doing, and the doing is mostly small decisions: how to split, what to compare against, which metric to trust, where to put the threshold. This page has ten exercises in two groups, graded from beginner to advanced. The first group is about regression and the basic workflow, the second about classification with a rare positive class. Every group starts with an editor that already holds the data.
How to use this page
Being stuck for a few minutes is where the learning happens.
What the ten exercises train
Regression first, then a rare-positive classification problem.
A few ground rules. The solutions use the same variable names as the editor in their group, and they build on each other: solution 1 creates the train and test split that solutions 2 to 5 use, so run them in order if you paste them under your own code. All data is generated with fixed seeds, so your numbers should match the output shown under each solution. If a model gets a slightly different score on your machine, that is normal; the conclusion should not change. Keep the lessons on Train, Validation and Test Sets, Regression Metrics and Classification Metrics at hand if you need a reminder.
Group 1: workflow and regression
The editor holds a small house-price table: area in square meters, rooms, age in years, and the target price in thousands. X is the feature table and y the target.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
1. A first split (Beginner)
Split X and y into a training part (75%) and a test part (25%) with random_state=0. Name the results X_train, X_test, y_train, y_test and print the shape of each.
Show solution
from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0) print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)
Output:
(225, 3) (75, 3) (225,) (75,)
train_test_split shuffles by default, so random_state is what makes your split reproducible. Everything you learn from the data from now on must come from X_train and y_train.
2. The baseline to beat (Beginner)
Before you fit a real model, fit a DummyRegressor that always predicts the training mean. Print its mean absolute error (MAE) and R squared on the test set. What R squared do you expect, and why?
Show solution
from sklearn.dummy import DummyRegressor from sklearn.metrics import mean_absolute_error, r2_score baseline = DummyRegressor(strategy="mean").fit(X_train, y_train) pred = baseline.predict(X_test) print("MAE:", round(mean_absolute_error(y_test, pred), 2)) print("R2 :", round(r2_score(y_test, pred), 3))
Output:
MAE: 88.23 R2 : -0.004
R squared compares a model with "always predict the mean of the data". The baseline predicts the mean of the training data, which is close to the mean of the test data but not identical, so R squared lands at or just below 0. Any real model has to beat this MAE.
3. Linear regression (Beginner)
Fit a LinearRegression on the training data. Print the intercept and the coefficient of each feature (with the feature names), and the MAE and R squared on the test set. Which feature has the largest coefficient? Is it the most important one?
Show solution
from sklearn.linear_model import LinearRegression model = LinearRegression().fit(X_train, y_train) print("intercept:", round(model.intercept_, 2)) for name, coef in zip(X.columns, model.coef_): print(f"{name:6s} {coef:7.2f}") pred = model.predict(X_test) print("MAE:", round(mean_absolute_error(y_test, pred), 2)) print("R2 :", round(r2_score(y_test, pred), 3))
Output:
intercept: 30.67 area 2.11 rooms 7.01 age -0.99 MAE: 15.32 R2 : 0.97
Here rooms has the largest coefficient and area a much smaller one, yet that does not make rooms more important. A coefficient depends on the unit of its feature: one more room is a huge change, one more square meter a tiny one. To compare importance you would standardize the features first (Multiple Linear Regression). Also note how far below the baseline's MAE this model is.
4. Metrics by hand (Intermediate)
Using only NumPy, compute the MAE and the root mean squared error (RMSE) of the linear model's test predictions, then check them against mean_absolute_error and root_mean_squared_error. Which of the two is larger, and why must that always hold?
Show solution
from sklearn.metrics import root_mean_squared_error errors = y_test.to_numpy() - pred mae = np.abs(errors).mean() rmse = np.sqrt((errors ** 2).mean()) print("by hand :", round(mae, 3), round(rmse, 3)) print("sklearn :", round(mean_absolute_error(y_test, pred), 3), round(root_mean_squared_error(y_test, pred), 3))
Output:
by hand : 15.316 18.525 sklearn : 15.316 18.525
RMSE is never smaller than MAE. Squaring punishes big errors more than small ones, so a few large misses pull RMSE up, while MAE treats every unit of error equally. The bigger the gap between them, the more your errors are dominated by a few large ones (Regression Metrics).
5. Compare with cross-validation (Intermediate)
Use 5-fold cross-validation on the training data, scored by R squared, to compare plain LinearRegression with a pipeline of StandardScaler and Ridge(alpha=10). Print the mean and standard deviation for both. Which is better, and is the difference meaningful?
Show solution
from sklearn.model_selection import cross_val_score, KFold from sklearn.pipeline import make_pipeline from sklearn.preprocessing import StandardScaler from sklearn.linear_model import Ridge cv = KFold(n_splits=5, shuffle=True, random_state=0) candidates = { "linear": LinearRegression(), "scaled ridge": make_pipeline(StandardScaler(), Ridge(alpha=10)), } for name, estimator in candidates.items(): scores = cross_val_score(estimator, X_train, y_train, cv=cv, scoring="r2") print(f"{name:13s} mean {scores.mean():.3f} std {scores.std():.3f}")
Output:
linear mean 0.974 std 0.009 scaled ridge mean 0.969 std 0.006
The two scores are almost identical, and the difference is far smaller than the spread between folds, so there is no meaningful winner. That is expected: with 3 features, 225 training rows and little multicollinearity, regularization has nothing to fix. The scaler sits inside the pipeline so each fold fits it on its own training part only (Pipelines).
Group 2: classification with a rare class
The editor generates a classification problem with six features where only about 10% of the samples are positive, like fraud or a rare disease. X is a NumPy array and y holds 0 and 1.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
6. A stratified split (Beginner)
Split into 70% train and 30% test with random_state=0, using stratify=y. Print the share of positives in y, in the training part and in the test part.
Show solution
from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, stratify=y, random_state=0) print("all :", round(y.mean(), 3)) print("train:", round(y_train.mean(), 3)) print("test :", round(y_test.mean(), 3))
Output:
all : 0.108 train: 0.109 test : 0.107
Stratification keeps the class shares the same in both parts. With a rare class and a small dataset, a plain random split can leave the test set with too few positives to measure anything.
7. Why accuracy fails (Beginner)
Fit a DummyClassifier(strategy="most_frequent"). Print its accuracy and its recall for the positive class on the test set, then explain in one sentence what this tells you about using accuracy on this data.
Show solution
from sklearn.dummy import DummyClassifier from sklearn.metrics import accuracy_score, recall_score dummy = DummyClassifier(strategy="most_frequent").fit(X_train, y_train) pred = dummy.predict(X_test) print("accuracy:", round(accuracy_score(y_test, pred), 3)) print("recall :", recall_score(y_test, pred))
Output:
accuracy: 0.893 recall : 0.0
A model that never finds a single positive scores about 90% accuracy, so accuracy cannot tell a useless model from a good one when one class dominates. Use recall, precision, or average precision instead (Imbalanced Classes).
8. A weighted logistic regression (Intermediate)
Build a pipeline with a StandardScaler and a LogisticRegression(class_weight="balanced"). Fit it and print the confusion matrix, the precision and the recall of the positive class on the test set. What does class_weight="balanced" do to the trade-off between precision and recall?
Show solution
from sklearn.pipeline import make_pipeline from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression from sklearn.metrics import confusion_matrix, precision_score clf = make_pipeline(StandardScaler(), LogisticRegression(class_weight="balanced")).fit(X_train, y_train) pred = clf.predict(X_test) print(confusion_matrix(y_test, pred)) print("precision:", round(precision_score(y_test, pred), 3)) print("recall :", round(recall_score(y_test, pred), 3))
Output:
[[219 49] [ 5 27]] precision: 0.355 recall : 0.844
Balanced weights make a mistake on a rare positive count as much as many mistakes on negatives. That pushes the model to flag more cases: recall rises a lot, and precision falls, because more of the flagged cases are false alarms. It is a way to move along the precision-recall trade-off, not a free improvement.
9. Choose a threshold for a recall target (Advanced)
Suppose the business requires catching at least 80% of the positives. Using the predicted probabilities of the model from exercise 8, find the highest threshold whose recall on the test set is at least 0.8, and print that threshold with the precision you get there. Use precision_recall_curve.
Show solution
from sklearn.metrics import precision_recall_curve proba = clf.predict_proba(X_test)[:, 1] precision, recall, thresholds = precision_recall_curve(y_test, proba) # precision and recall have one more entry than thresholds, so drop the last one ok = recall[:-1] >= 0.8 best = np.flatnonzero(ok)[-1] print("threshold:", round(thresholds[best], 3)) print("recall :", round(recall[best], 3)) print("precision:", round(precision[best], 3))
Output:
threshold: 0.592 recall : 0.812 precision: 0.473
Recall falls as the threshold rises, so the last index that still satisfies the target belongs to the highest acceptable threshold, and it has the best precision among the options that meet the target. In a real project you would pick this threshold on validation data, not on the test set that you also use to report results.
10. Scaling and nearest neighbors (Advanced)
k-nearest neighbors depends on distances. Make a copy of the training and test features where column 0 is multiplied by 1000 (as if it were measured in a smaller unit). Fit KNeighborsClassifier(n_neighbors=5) on the stretched data, with and without a StandardScaler, and compare the balanced accuracy on the test set. Explain the result.
Show solution
from sklearn.neighbors import KNeighborsClassifier from sklearn.metrics import balanced_accuracy_score X_train_big, X_test_big = X_train.copy(), X_test.copy() X_train_big[:, 0] *= 1000 X_test_big[:, 0] *= 1000 plain = KNeighborsClassifier(n_neighbors=5).fit(X_train_big, y_train) scaled = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=5)).fit(X_train_big, y_train) print("stretched, no scaling:", round(balanced_accuracy_score(y_test, plain.predict(X_test_big)), 3)) print("stretched, scaled :", round(balanced_accuracy_score(y_test, scaled.predict(X_test_big)), 3))
Output:
stretched, no scaling: 0.526 stretched, scaled : 0.743
With column 0 a thousand times larger, it alone decides every distance, so the neighbors are chosen by a single feature and the other five are ignored. Standardizing restores equal weight, and the score recovers. The pipeline also learns the scaling from the training part only. Balanced accuracy (the average of recall over both classes) is used because plain accuracy would hide the problem on this lopsided data.
Where to go next
If these felt comfortable, continue with Exercises: Ensembles, Pipelines and Unsupervised. If a step felt shaky, return to the matching lesson: Baselines and Dummy Models, Thresholds, ROC and Precision-Recall Curves or k-Nearest Neighbors.
Summary
- Split first and learn only from the training part; stratify when a class is rare.
- Every model needs a baseline, and accuracy is a poor metric when one class dominates.
- Compare models with cross-validation and judge the gap against the spread between folds.
- RMSE is never below MAE; the gap shows how much a few large errors dominate.
- Class weights and thresholds move you along the precision-recall trade-off, so choose them from a stated goal.
- Distance-based models need scaled features, and the scaler belongs inside a pipeline.
Continue with Exercises: Ensembles, Pipelines and Unsupervised.
Track your progress
Create a free account to tick off lessons and pick up where you left off on any device.
of Machine Learning lessons completed