Skip to content
Tutorials Center
My learning

Topics

Lesson 107 of 111 ยท Practice and Review

Exercises: Workflow, Regression and Classification

Ten graded scikit-learn exercises with full solutions: splits, baselines, linear regression, metrics, imbalanced classification, thresholds and feature scaling.

Machine learning is learned by doing, and the doing is mostly small decisions: how to split, what to compare against, which metric to trust, where to put the threshold. This page has ten exercises in two groups, graded from beginner to advanced. The first group is about regression and the basic workflow, the second about classification with a rare positive class. Every group starts with an editor that already holds the data.

How to use this page

Try first Write your own answer in the editor.
Run it Print intermediate results.
Peek Open the solution and compare.
Vary it Change a number and predict the result.

Being stuck for a few minutes is where the learning happens.

1 / 2

A few ground rules. The solutions use the same variable names as the editor in their group, and they build on each other: solution 1 creates the train and test split that solutions 2 to 5 use, so run them in order if you paste them under your own code. All data is generated with fixed seeds, so your numbers should match the output shown under each solution. If a model gets a slightly different score on your machine, that is normal; the conclusion should not change. Keep the lessons on Train, Validation and Test Sets, Regression Metrics and Classification Metrics at hand if you need a reminder.

Group 1: workflow and regression

The editor holds a small house-price table: area in square meters, rooms, age in years, and the target price in thousands. X is the feature table and y the target.

Python Group 1 workspace: houses

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

1. A first split (Beginner)

Split X and y into a training part (75%) and a test part (25%) with random_state=0. Name the results X_train, X_test, y_train, y_test and print the shape of each.

Show solution
Python
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)

Output:

TEXT
(225, 3) (75, 3) (225,) (75,)

train_test_split shuffles by default, so random_state is what makes your split reproducible. Everything you learn from the data from now on must come from X_train and y_train.

2. The baseline to beat (Beginner)

Before you fit a real model, fit a DummyRegressor that always predicts the training mean. Print its mean absolute error (MAE) and R squared on the test set. What R squared do you expect, and why?

Show solution
Python
from sklearn.dummy import DummyRegressor
from sklearn.metrics import mean_absolute_error, r2_score

baseline = DummyRegressor(strategy="mean").fit(X_train, y_train)
pred = baseline.predict(X_test)
print("MAE:", round(mean_absolute_error(y_test, pred), 2))
print("R2 :", round(r2_score(y_test, pred), 3))

Output:

TEXT
MAE: 88.23
R2 : -0.004

R squared compares a model with "always predict the mean of the data". The baseline predicts the mean of the training data, which is close to the mean of the test data but not identical, so R squared lands at or just below 0. Any real model has to beat this MAE.

3. Linear regression (Beginner)

Fit a LinearRegression on the training data. Print the intercept and the coefficient of each feature (with the feature names), and the MAE and R squared on the test set. Which feature has the largest coefficient? Is it the most important one?

Show solution
Python
from sklearn.linear_model import LinearRegression

model = LinearRegression().fit(X_train, y_train)
print("intercept:", round(model.intercept_, 2))
for name, coef in zip(X.columns, model.coef_):
    print(f"{name:6s} {coef:7.2f}")
pred = model.predict(X_test)
print("MAE:", round(mean_absolute_error(y_test, pred), 2))
print("R2 :", round(r2_score(y_test, pred), 3))

Output:

TEXT
intercept: 30.67
area      2.11
rooms     7.01
age      -0.99
MAE: 15.32
R2 : 0.97

Here rooms has the largest coefficient and area a much smaller one, yet that does not make rooms more important. A coefficient depends on the unit of its feature: one more room is a huge change, one more square meter a tiny one. To compare importance you would standardize the features first (Multiple Linear Regression). Also note how far below the baseline's MAE this model is.

4. Metrics by hand (Intermediate)

Using only NumPy, compute the MAE and the root mean squared error (RMSE) of the linear model's test predictions, then check them against mean_absolute_error and root_mean_squared_error. Which of the two is larger, and why must that always hold?

Show solution
Python
from sklearn.metrics import root_mean_squared_error

errors = y_test.to_numpy() - pred
mae = np.abs(errors).mean()
rmse = np.sqrt((errors ** 2).mean())
print("by hand  :", round(mae, 3), round(rmse, 3))
print("sklearn  :", round(mean_absolute_error(y_test, pred), 3), round(root_mean_squared_error(y_test, pred), 3))

Output:

TEXT
by hand  : 15.316 18.525
sklearn  : 15.316 18.525

RMSE is never smaller than MAE. Squaring punishes big errors more than small ones, so a few large misses pull RMSE up, while MAE treats every unit of error equally. The bigger the gap between them, the more your errors are dominated by a few large ones (Regression Metrics).

5. Compare with cross-validation (Intermediate)

Use 5-fold cross-validation on the training data, scored by R squared, to compare plain LinearRegression with a pipeline of StandardScaler and Ridge(alpha=10). Print the mean and standard deviation for both. Which is better, and is the difference meaningful?

Show solution
Python
from sklearn.model_selection import cross_val_score, KFold
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge

cv = KFold(n_splits=5, shuffle=True, random_state=0)
candidates = {
    "linear": LinearRegression(),
    "scaled ridge": make_pipeline(StandardScaler(), Ridge(alpha=10)),
}
for name, estimator in candidates.items():
    scores = cross_val_score(estimator, X_train, y_train, cv=cv, scoring="r2")
    print(f"{name:13s} mean {scores.mean():.3f}  std {scores.std():.3f}")

Output:

TEXT
linear        mean 0.974  std 0.009
scaled ridge  mean 0.969  std 0.006

The two scores are almost identical, and the difference is far smaller than the spread between folds, so there is no meaningful winner. That is expected: with 3 features, 225 training rows and little multicollinearity, regularization has nothing to fix. The scaler sits inside the pipeline so each fold fits it on its own training part only (Pipelines).

Group 2: classification with a rare class

The editor generates a classification problem with six features where only about 10% of the samples are positive, like fraud or a rare disease. X is a NumPy array and y holds 0 and 1.

Python Group 2 workspace: rare positives

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

6. A stratified split (Beginner)

Split into 70% train and 30% test with random_state=0, using stratify=y. Print the share of positives in y, in the training part and in the test part.

Show solution
Python
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, stratify=y, random_state=0)
print("all  :", round(y.mean(), 3))
print("train:", round(y_train.mean(), 3))
print("test :", round(y_test.mean(), 3))

Output:

TEXT
all  : 0.108
train: 0.109
test : 0.107

Stratification keeps the class shares the same in both parts. With a rare class and a small dataset, a plain random split can leave the test set with too few positives to measure anything.

7. Why accuracy fails (Beginner)

Fit a DummyClassifier(strategy="most_frequent"). Print its accuracy and its recall for the positive class on the test set, then explain in one sentence what this tells you about using accuracy on this data.

Show solution
Python
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, recall_score

dummy = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
pred = dummy.predict(X_test)
print("accuracy:", round(accuracy_score(y_test, pred), 3))
print("recall  :", recall_score(y_test, pred))

Output:

TEXT
accuracy: 0.893
recall  : 0.0

A model that never finds a single positive scores about 90% accuracy, so accuracy cannot tell a useless model from a good one when one class dominates. Use recall, precision, or average precision instead (Imbalanced Classes).

8. A weighted logistic regression (Intermediate)

Build a pipeline with a StandardScaler and a LogisticRegression(class_weight="balanced"). Fit it and print the confusion matrix, the precision and the recall of the positive class on the test set. What does class_weight="balanced" do to the trade-off between precision and recall?

Show solution
Python
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix, precision_score

clf = make_pipeline(StandardScaler(), LogisticRegression(class_weight="balanced")).fit(X_train, y_train)
pred = clf.predict(X_test)
print(confusion_matrix(y_test, pred))
print("precision:", round(precision_score(y_test, pred), 3))
print("recall   :", round(recall_score(y_test, pred), 3))

Output:

TEXT
[[219  49]
 [  5  27]]
precision: 0.355
recall   : 0.844

Balanced weights make a mistake on a rare positive count as much as many mistakes on negatives. That pushes the model to flag more cases: recall rises a lot, and precision falls, because more of the flagged cases are false alarms. It is a way to move along the precision-recall trade-off, not a free improvement.

9. Choose a threshold for a recall target (Advanced)

Suppose the business requires catching at least 80% of the positives. Using the predicted probabilities of the model from exercise 8, find the highest threshold whose recall on the test set is at least 0.8, and print that threshold with the precision you get there. Use precision_recall_curve.

Show solution
Python
from sklearn.metrics import precision_recall_curve

proba = clf.predict_proba(X_test)[:, 1]
precision, recall, thresholds = precision_recall_curve(y_test, proba)
# precision and recall have one more entry than thresholds, so drop the last one
ok = recall[:-1] >= 0.8
best = np.flatnonzero(ok)[-1]
print("threshold:", round(thresholds[best], 3))
print("recall   :", round(recall[best], 3))
print("precision:", round(precision[best], 3))

Output:

TEXT
threshold: 0.592
recall   : 0.812
precision: 0.473

Recall falls as the threshold rises, so the last index that still satisfies the target belongs to the highest acceptable threshold, and it has the best precision among the options that meet the target. In a real project you would pick this threshold on validation data, not on the test set that you also use to report results.

10. Scaling and nearest neighbors (Advanced)

k-nearest neighbors depends on distances. Make a copy of the training and test features where column 0 is multiplied by 1000 (as if it were measured in a smaller unit). Fit KNeighborsClassifier(n_neighbors=5) on the stretched data, with and without a StandardScaler, and compare the balanced accuracy on the test set. Explain the result.

Show solution
Python
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import balanced_accuracy_score

X_train_big, X_test_big = X_train.copy(), X_test.copy()
X_train_big[:, 0] *= 1000
X_test_big[:, 0] *= 1000

plain = KNeighborsClassifier(n_neighbors=5).fit(X_train_big, y_train)
scaled = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=5)).fit(X_train_big, y_train)
print("stretched, no scaling:", round(balanced_accuracy_score(y_test, plain.predict(X_test_big)), 3))
print("stretched, scaled    :", round(balanced_accuracy_score(y_test, scaled.predict(X_test_big)), 3))

Output:

TEXT
stretched, no scaling: 0.526
stretched, scaled    : 0.743

With column 0 a thousand times larger, it alone decides every distance, so the neighbors are chosen by a single feature and the other five are ignored. Standardizing restores equal weight, and the score recovers. The pipeline also learns the scaling from the training part only. Balanced accuracy (the average of recall over both classes) is used because plain accuracy would hide the problem on this lopsided data.

Where to go next

If these felt comfortable, continue with Exercises: Ensembles, Pipelines and Unsupervised. If a step felt shaky, return to the matching lesson: Baselines and Dummy Models, Thresholds, ROC and Precision-Recall Curves or k-Nearest Neighbors.

Summary

  • Split first and learn only from the training part; stratify when a class is rare.
  • Every model needs a baseline, and accuracy is a poor metric when one class dominates.
  • Compare models with cross-validation and judge the gap against the spread between folds.
  • RMSE is never below MAE; the gap shows how much a few large errors dominate.
  • Class weights and thresholds move you along the precision-recall trade-off, so choose them from a stated goal.
  • Distance-based models need scaled features, and the scaler belongs inside a pipeline.

Continue with Exercises: Ensembles, Pipelines and Unsupervised.