Skip to content
Tutorials Center
My learning

Topics

Lesson 70 of 76 ยท Practice and Review

Practice Exercises with Solutions

22 graded data science exercises with worked solutions: cleaning, exploratory analysis, statistics and modelling. Practice in your browser with ready data.

Data science is a craft, and you learn a craft by doing. This page has 22 exercises in four groups that follow the order of a real project: cleaning, exploratory analysis, statistics, and modelling. Every group starts with an editor that already contains the data, so you can concentrate on the question. Inside each group the exercises go from beginner to advanced, and every one has a worked solution you can open after your own attempt.

A few rules make the practice work:

  • Predict before you run. Write down what you expect the output to look like. The gap between expectation and result is where you learn.
  • Try first, peek later. Struggle for a few minutes before opening a solution. The solution shows one good way; yours may differ and still be right.
  • Work in the editor. Solutions in a group build on the variables of earlier ones, so run them in order. Paste a solution under the # Exercise N marker to try it.
  • Say the result in words. For every statistical or modelling answer, write one sentence a manager could understand. A number without a sentence is half an answer.
flowchart LR
    A["Read the task"] --> B["Predict the output"]
    B --> C["Write and run"]
    C --> D{"Matches the<br/>expected result?"}
    D -- "yes" --> E["Open the solution<br/>and compare"]
    D -- "no" --> F["Print the intermediate<br/>steps and fix"]
    F --> C
    E --> G["Write the result<br/>in one sentence"]

The practice loop. Printing intermediate steps is the habit that pays off in every real project.

Keep the cheat sheet open in another tab if you need a reminder.

Group 1: cleaning

A small export of online orders with the usual problems: inconsistent country names, money stored as text, an impossible date, a text that is not a date, a double submission and a missing amount. These exercises practice ideas from Auditing Data Quality, Handling Missing Values and Cleaning Text and Categories. Everything starts as text (dtype=str), so each fix is explicit.

Python Group 1 workspace: orders

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

1. One name per country (Beginner)

The country column has pt, PT , Portugal, norway, NO and so on. Convert it to the codes PT, NO and EG and count the orders per country. Expected: PT 5, NO 3, EG 2.

Show solution
Python
codes = {"pt": "PT", "portugal": "PT", "no": "NO", "norway": "NO", "eg": "EG", "egypt": "EG"}
orders["country"] = orders["country"].str.strip().str.lower().map(codes)
print(orders["country"].value_counts().to_string())

Normalize the text first (strip spaces, lowercase) and then map through an explicit dictionary. A spelling you did not list becomes NaN, which is better than a silent wrong category.

2. Money as numbers (Beginner)

Turn amount ("$1,200.50") into a float column. Print the total and the number of missing amounts. Expected: a total of 3877.84 and 1 missing value.

Show solution
Python
orders["amount"] = pd.to_numeric(orders["amount"].str.replace(r"[$,]", "", regex=True))
print(orders["amount"].sum(), orders["amount"].isna().sum())

The regular expression removes the dollar sign and the thousands separators. An empty cell is already NaN, and sum() skips it. The printed total is 3877.84 (floating-point arithmetic may show a long tail of digits, so round when you report it).

3. Find the bad dates (Intermediate)

Parse date as a date without raising an error, then list the order_id of every row whose date could not be parsed. Expected: ['1004', '1006'].

Show solution
Python
orders["date"] = pd.to_datetime(orders["date"], errors="coerce")
print(orders.loc[orders["date"].isna(), "order_id"].tolist())

errors="coerce" turns anything unparseable into NaT (the date version of NaN). Order 1004 has February 30 and order 1006 has plain text. Listing the failures, instead of silently dropping them, is how you decide whether to fix, ask or exclude them.

4. Spot the double submission (Intermediate)

Standardize customer (strip spaces, title case), report how many distinct customers there are, then find rows that repeat the same customer, amount and date. Drop the later copy. Expected: 8 customers, order 1009 is the duplicate, 9 rows remain.

Show solution
Python
orders["customer"] = orders["customer"].str.strip().str.title()
print(orders["customer"].nunique())
dup = orders.duplicated(subset=["customer", "amount", "date"], keep="first")
print(orders.loc[dup, "order_id"].tolist())
orders = orders[~dup]
print(len(orders))

Without standardizing the names, ben ode and Ben Ode would look like different people and the duplicate would stay hidden. duplicated(..., keep="first") marks every repeat after the first occurrence.

5. Fill the missing amount without peeking (Advanced)

Order 1005 has no amount. Fill it with the median amount of its own country, using the cleaned table. Expected: 192.55 for order 1005 and no missing amounts left.

Show solution
Python
medians = orders.groupby("country")["amount"].transform("median")
orders["amount"] = orders["amount"].fillna(medians)
print(orders.loc[orders["order_id"] == "1005", ["order_id", "country", "amount"]].to_string(index=False))
print(orders["amount"].isna().sum())

transform("median") returns a Series aligned with every row, so fillna can use it directly. The two other Norwegian orders cost 310.00 and 75.10, and the median of those is 192.55. In a modelling project you would compute such medians on the training rows only, to avoid leakage.

Group 2: exploratory analysis

A table of 240 sales orders with region, product, units, price, discount and revenue, generated with a fixed seed so everyone gets the same numbers. These exercises practice a systematic EDA process, single variables and relationships.

Python Group 2 workspace: sales

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

6. Compare regions (Beginner)

For each region show the number of orders, the mean revenue and the median revenue. Which region has the highest mean? Expected: West has the highest mean (337.8) and East the lowest (306.6).

Show solution
Python
print(sales.groupby("region")["revenue"].agg(["count", "mean", "median"]).round(1))

Always look at the count next to the average, and the median next to the mean. Here the means differ by about 10 percent but each region has only 54 to 68 orders, so do not rush to call West "the best region" before testing it.

7. Skew and a log transform (Beginner)

Compute the skewness of revenue and of log(revenue). Expected: 1.06 and -0.15.

Show solution
Python
print(round(sales["revenue"].skew(), 2), round(np.log(sales["revenue"]).skew(), 2))

Revenue is right-skewed (a few large orders pull the mean up). The logarithm makes the distribution nearly symmetric, which is why log transforms are so common for money and counts.

8. A suspicious correlation (Intermediate)

Compute the correlation between units and price across all orders, then inside each product. Explain the difference. Expected: -0.68 overall; about -0.18 (Basic), -0.07 (Plus) and 0.06 (Pro) inside products.

Show solution
Python
print(round(sales["units"].corr(sales["price"]), 2))
print(sales.groupby("product").apply(lambda g: g["units"].corr(g["price"]), include_groups=False).round(2).to_string())

The strong overall correlation comes from the mix of products: cheap Basic items sell in large quantities and expensive Pro items in small ones. Inside a product, price barely varies and the relationship almost disappears. This is a close cousin of Simpson's paradox: always check whether a pattern survives inside groups.

9. Two charts (Intermediate)

Draw a histogram of revenue (30 bins) next to a box plot of revenue by product (Basic, Plus, Pro), with titles and axis labels.

Show solution
Python
fig, axes = plt.subplots(1, 2, figsize=(9, 3.4))
axes[0].hist(sales["revenue"], bins=30)
axes[0].set(title="Revenue per order", xlabel="Revenue", ylabel="Orders")
order = ["Basic", "Plus", "Pro"]
axes[1].boxplot([sales.loc[sales["product"] == p, "revenue"] for p in order], tick_labels=order)
axes[1].set(title="Revenue by product", ylabel="Revenue")
fig.tight_layout()
plt.show()

The histogram shows the long right tail; the box plots show that the Pro product produces most of the high-revenue orders. Passing the categories in a fixed list makes the box order deliberate instead of alphabetical.

10. The IQR outlier rule (Intermediate)

Compute the upper fence Q3 + 1.5 * IQR for revenue, count the orders above it, and say which products they come from. Expected: a fence of about 655.3, 11 orders, 9 of them Pro and 2 Plus.

Show solution
Python
q1, q3 = sales["revenue"].quantile([0.25, 0.75])
upper = q3 + 1.5 * (q3 - q1)
print(round(upper, 1), int((sales["revenue"] > upper).sum()))
print(sales.loc[sales["revenue"] > upper, "product"].value_counts().to_string())

The flagged orders are not errors; they are simply big Pro orders. A statistical rule only nominates candidates. The decision to remove, cap or keep them needs a reason from the domain (Outliers, Errors and Impossible Values).

11. The best product per region (Advanced)

Build a table of total revenue with regions as rows and products as columns, then print the product with the highest total for each region. Expected: East and West lead with Basic, North and South with Plus.

Show solution
Python
totals = sales.pivot_table(index="region", columns="product", values="revenue", aggfunc="sum").round(0)
print(totals)
print(totals.idxmax(axis=1).to_string())

pivot_table reshapes long data into a grid, and idxmax(axis=1) returns the column label of the largest value in each row. Notice that the winner is not Pro anywhere even though Pro has the highest order values: Pro has few orders.

Group 3: statistics

Here the data is two samples of checkout times in seconds (old and new design) and a conversion table. These exercises use ideas from Estimation and Confidence Intervals, Hypothesis Testing Done Right and A/B Testing and Experiments.

Python Group 3 workspace: checkout times

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

12. A confidence interval (Beginner)

Compute the mean checkout time of the old design (old_flow) and a 95 percent t-based confidence interval. Expected: mean 50.0 and an interval from 45.5 to 54.5 seconds.

Show solution
Python
m, se = old_flow.mean(), stats.sem(old_flow)
low, high = stats.t.interval(0.95, df=len(old_flow) - 1, loc=m, scale=se)
print(round(m, 1), round(low, 1), round(high, 1))

stats.sem is the standard error, stats.t.interval turns it into a range using the t distribution with n - 1 degrees of freedom. The sentence: "If we repeated this study many times, intervals built this way would contain the true mean about 95 percent of the time."

13. Old design vs new design (Intermediate)

Compare old_flow and new_flow with Welch's t-test and compute Cohen's d. Interpret the result. Expected: a 4.2-second difference, t = 1.41, p = 0.1618, d = 0.31.

Show solution
Python
t, p = stats.ttest_ind(old_flow, new_flow, equal_var=False)
pooled = np.sqrt((old_flow.var(ddof=1) + new_flow.var(ddof=1)) / 2)
d = (old_flow.mean() - new_flow.mean()) / pooled
print(round(old_flow.mean() - new_flow.mean(), 1), round(t, 2), round(p, 4), round(d, 2))

A p-value of 0.16 does not show the designs are equal. It says that a difference this size is quite possible by chance with 40 and 45 users. The effect size d = 0.31 is small to moderate, so the study is simply too small to tell. The right report is "the new design may be a few seconds faster, but we need more data".

14. Did conversion change? (Intermediate)

The table [[48, 352], [75, 325]] holds bought / did not buy for the old and new checkout (400 users each). Compute both conversion rates and a chi-square test. Expected: 12.0 percent and 18.75 percent, chi-square 7.0, p = 0.0081.

Show solution
Python
table = np.array([[48, 352], [75, 325]])      # rows: old, new; columns: bought, did not buy
chi2, p_chi, dof, expected = stats.chi2_contingency(table, correction=False)
print(table[:, 0] / table.sum(axis=1), round(chi2, 2), round(p_chi, 4))

This time the evidence is strong: p = 0.0081. Notice the contrast with exercise 13. A comparison of proportions with 400 users per group found a real difference of 6.75 percentage points, while the comparison of times with about 40 per group did not. Sample size and effect size decide what you can detect.

15. How big must the next test be? (Advanced)

You want to detect a rise in conversion from 10 percent to 12 percent with 80 percent power at the 5 percent significance level. How many users per group do you need? Expected: effect size 0.064 and 3,835 users per group.

Show solution
Python
effect = proportion_effectsize(0.12, 0.10)
n_per_group = NormalIndPower().solve_power(effect, alpha=0.05, power=0.8, alternative="two-sided")
print(round(effect, 3), int(np.ceil(n_per_group)))

Small effects need big samples: roughly 3,800 users in each group for a two-point lift. If your site only gets 500 users a week, this experiment would take months, and that is worth knowing before you start.

16. A permutation test (Advanced)

Test the difference in mean checkout times from exercise 13 without any distribution assumptions: shuffle the pooled values 5,000 times, split them into two groups of the original sizes, and count how often the shuffled difference is at least as large as the observed one. Expected: an observed difference of about 4.18 and a p-value near 0.15.

Show solution
Python
rng2 = np.random.default_rng(0)
pooled_all = np.concatenate([old_flow, new_flow])
observed = old_flow.mean() - new_flow.mean()
count = 0
reps = 5000
for _ in range(reps):
    rng2.shuffle(pooled_all)
    diff = pooled_all[:len(old_flow)].mean() - pooled_all[len(old_flow):].mean()
    count += abs(diff) >= abs(observed)
print(round(observed, 2), (count + 1) / (reps + 1))

If the designs made no difference, the labels "old" and "new" would be arbitrary, so shuffling them shows what chance alone produces. The permutation p-value (about 0.15) agrees with Welch's 0.16. Adding 1 to numerator and denominator avoids reporting a p-value of exactly zero.

17. Twenty tests on pure noise (Advanced)

Run 20 t-tests, each comparing two groups of 30 values drawn from the same normal distribution (use np.random.default_rng(4)). Count how many p-values fall below 0.05, and below the Bonferroni threshold 0.05 / 20. Expected: 3 and 0, smallest p-value about 0.036.

Show solution
Python
rng3 = np.random.default_rng(4)
pvals = [stats.ttest_ind(rng3.normal(0, 1, 30), rng3.normal(0, 1, 30)).pvalue for _ in range(20)]
pvals = np.array(pvals)
print((pvals < 0.05).sum(), (pvals < 0.05 / 20).sum(), round(pvals.min(), 3))

Even though nothing is going on, three of twenty tests look "significant" at 0.05 (on average one in twenty would). Slice a dataset enough ways and something will pass; the Bonferroni threshold corrects for that. This is how p-hacking produces false discoveries.

Group 4: modelling

A customer table with tenure_months, support_calls, monthly_spend, a column called exit_survey_sent, and the target churned (1 = left). About a quarter of the customers churn. These exercises use Classification with Logistic Regression, Train/Test Splits and Cross-Validation and Evaluating Models. The solutions in this group are meant to be run in order.

Python Group 4 workspace: churn

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

18. The baseline (Beginner)

What share of customers churn, and what accuracy would a model get by always predicting "stays"? Expected: 0.253 churn, so 0.747 accuracy for the lazy rule.

Show solution
Python
print(round(data["churned"].mean(), 3), round(1 - data["churned"].mean(), 3))

The majority-class accuracy is the number to beat. With imbalanced classes, an accuracy of 75 percent can mean the model learned nothing.

19. A cross-validated logistic regression (Intermediate)

Using only the three legitimate features, estimate the ROC AUC of a scaled logistic regression with stratified 5-fold cross-validation. Expected: a mean AUC of 0.721 with a standard deviation of 0.06.

Show solution
Python
cv = StratifiedKFold(5, shuffle=True, random_state=0)
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
auc = cross_val_score(model, data[features], data["churned"], cv=cv, scoring="roc_auc")
print(round(auc.mean(), 3), round(auc.std(), 3))

AUC 0.5 is a coin flip and 1.0 is perfect, so 0.72 is a modest but real signal. Putting the scaler inside the pipeline means each fold scales using only its own training rows. The spread of 0.06 across folds is a reminder that a single split would give a noisy answer.

20. Too good to be true (Intermediate)

Add exit_survey_sent to the features and repeat the cross-validation. Expected: AUC jumps to 0.961. Explain why this is a warning sign, not a success.

Show solution
Python
auc_leaky = cross_val_score(model, data[features + ["exit_survey_sent"]], data["churned"], cv=cv, scoring="roc_auc")
print(round(auc_leaky.mean(), 3))

An exit survey is sent after a customer leaves, so the column is a consequence of the target, not a cause. This is target leakage: the model looks brilliant in cross-validation and would be useless in production, where the survey does not exist yet at prediction time. When a score jumps dramatically, suspect leakage first and ask for every feature: would I know this at the moment of prediction?

21. Choose a threshold (Advanced)

Split the data (25 percent test, stratified, random_state=1), fit the pipeline on the training part, and on the test part find the largest threshold that still gives at least 80 percent recall. Report the threshold, the recall, the precision and the share of customers flagged. Expected: threshold 0.207, recall 0.82, precision 0.39, 53 percent flagged.

Show solution
Python
X_train, X_test, y_train, y_test = train_test_split(
    data[features], data["churned"], test_size=0.25, stratify=data["churned"], random_state=1)
model.fit(X_train, y_train)
proba = model.predict_proba(X_test)[:, 1]
precision, recall, thresholds = precision_recall_curve(y_test, proba)
ok = recall[:-1] >= 0.8
best = np.where(ok)[0][-1]
print(round(thresholds[best], 3), round(recall[best], 2), round(precision[best], 2))
print("flagged share:", round((proba >= thresholds[best]).mean(), 2))

Recall falls as the threshold rises, so the last threshold that keeps recall at 0.8 is the highest one allowed. The price: with this weak model we must flag more than half of all customers to catch 82 percent of the churners, and only 39 percent of flagged customers actually leave. Whether that is acceptable depends on the cost of a retention offer compared with the value of a saved customer. (Choosing the threshold on the same test set you report on is optimistic; use validation data for the choice in real work.)

22. What the default threshold hides (Advanced)

At the default threshold of 0.5, build the confusion matrix on the test set and compare accuracy with the majority-class baseline. Expected: accuracy 0.767 against a baseline of 0.747, and recall of only 0.16.

Show solution
Python
pred = (proba >= 0.5).astype(int)
confusion = pd.crosstab(y_test.rename("actual"), pd.Series(pred, index=y_test.index, name="predicted"))
print(confusion)
print("accuracy:", round((pred == y_test).mean(), 3), " majority baseline:", round(1 - y_test.mean(), 3))
print("recall at 0.5:", round(pred[y_test == 1].mean(), 2))

The matrix shows only 6 of the 38 churners caught. Accuracy looks respectable (77 percent) and barely beats "always predict stays" (75 percent). With imbalanced classes, accuracy and the default threshold hide the thing you care about, which is why exercise 21 chose its threshold from the cost of errors.

What to practice next

If you finished everything, change the setup and repeat: edit the generators (different seeds, more noise, fewer rows) and see which conclusions survive. Then try a full project: House Price Model or Survey Analysis. If you struggled with a group, go back to its chapter, then redo the exercises a few days later without peeking at the solutions.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A model reaches 77 percent accuracy when 75 percent of customers do not churn. What should you conclude?

    With imbalanced classes, accuracy hides failure on the minority class. Compare to the baseline and examine recall and precision.
  2. 2. Why does a p-value of 0.16 not prove two designs are equal?

    Absence of evidence is not evidence of absence. A small study can fail to detect a real difference; report the effect size and interval.
  3. 3. A feature raises cross-validated AUC from 0.72 to 0.96. What is the first thing to check?

    Sudden near-perfect scores usually mean the feature contains information from the future or from the target itself.
  4. 4. Twenty tests on pure noise produce three p-values below 0.05. What does this illustrate?

    At the 0.05 level, about one test in twenty passes by chance. A correction such as Bonferroni lowers the threshold in proportion to the number of tests.

Summary

  • Practice in the order of a project: clean, explore, test, model; predict each output before you run it.
  • Normalize text, then map through an explicit dictionary; list parse failures instead of hiding them.
  • Always look at counts, medians and groups next to averages and overall correlations.
  • Report effect sizes and intervals, not just p-values; small samples can fail to detect real effects.
  • Beware multiple comparisons and target leakage: both produce results that look better than they are.
  • Compare models to a baseline and choose thresholds from the costs of errors, not from the default 0.5.

Next: Common Data Science Mistakes, a gallery of the errors behind many wrong conclusions.