Skip to content
Tutorials Center
My learning

Topics

Lesson 68 of 76 ยท Projects

Projects: Wine, Fraud Detection, Loan Approval and Heart Disease

Four compact classification projects with scikit-learn: wine classes, imbalanced fraud detection, loan approval with a fairness check, and a heart-disease risk model with calibration.

The best way to cement modelling skills is to repeat the same loop on different problems and notice what changes. Each project below follows the same routine: frame the question, split first, start with a baseline, fit a model in a pipeline, evaluate with the right metric and say what the result means. What changes from project to project is the thing that makes each problem tricky:

Project The twist
Wine Clean numeric data, several classes: a baseline for what "good" looks like
Fraud detection Extreme class imbalance: accuracy is useless
Loan approval Mixed column types and a decision that affects people: fairness checks
Heart disease A risk score: calibration and interpretability matter

Three of the datasets are synthetic, generated with a fixed seed, because the well-known public versions need a download and some contain real patients. The wine data is a small real dataset that ships with scikit-learn. Because we generate the synthetic data, we also know the rules behind it, which lets you check whether a model recovers them. Real data never offers that luxury. The background is in Train, Test and Validation, Evaluating Models and Pipelines and Tuning.

Project 1: Wine cultivars

Question: can the chemical analysis of a wine tell which of three grape cultivars produced it? The data (load_wine) has 178 wines with 13 measurements such as alcohol, flavanoids and colour intensity.

Small, clean data is the right place to learn the baseline habit. We compare a dummy model (always the most common class) with a scaled logistic regression and a random forest, using cross-validation so that 178 rows are used well.

Python Wine: baseline, two models and the confusion matrix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The baseline sits near 40% because the classes are close to balanced, so any real model clearly beats it. The two models reach high accuracy, and the confusion matrix shows that the few errors involve neighbouring classes. A very high score on a small, clean dataset is not a promise for messy production data, so report the cross-validated spread, not one lucky split.

Project 2: Fraud detection

Question: which card transactions are fraudulent? The twist is the base rate: in real data fewer than 1% of transactions are fraud. A model that says "legitimate" for every transaction is 99%+ accurate and catches nothing. We need metrics that ignore the easy majority: precision (of the transactions we flag, how many are fraud?), recall (of all fraud, how much do we catch?) and average precision (the area under the precision-recall curve).

Python Fraud: accuracy lies, precision and recall tell the truth

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The useless baseline has the highest accuracy of all (99.3 percent, against about 98.9 for the real models), yet the models catch roughly 40 to 50 percent of the fraud while a reviewer looks at only 1 percent of transactions. Average precision (about 0.25, against a base rate of 0.7 percent, so more than 30 times better than chance) and recall show what accuracy hides. The signal in this synthetic data is modest on purpose: real fraud models also live with low precision. The threshold is a business choice: reviewing the top 1% of transactions is a staffing decision, and the model's job is to put as many frauds as possible in that 1%. With imbalanced data also keep the stratified split (done above), and use class_weight or resampling only on the training set. See Evaluating Models for more on metrics.

Project 3: Loan approval

Question: should this application be approved? The new difficulties: columns of mixed types (numbers and categories, some missing) and a decision that changes people's lives, so we must check that the model does not treat groups unfairly. We use a ColumnTransformer to prepare each column type, and then compare approval rates and error rates across a sensitive group.

Python Loan approval: a mixed-type pipeline and a group check

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The sensitive column group is not a model input, yet we still compare outcomes by group. Removing a protected attribute does not guarantee fair results, because other columns can act as proxies for it, so you measure. Here the rule that created the data ignores the group, so the approval rates, the share of good applicants who are wrongly rejected and the AUC should be similar. If they were not, the next step would be to investigate which features carry the difference, not to hide it. Read Ethics and Fairness before using a model like this for real decisions, and note that lending is a regulated area where explanations are often a legal requirement.

Project 4: Heart disease risk

Question: how likely is a patient to have heart disease, given age, blood pressure, cholesterol, smoking and exercise? In medicine a probability matters more than a yes/no label, so two properties are essential: the model should be interpretable (a doctor asks "why?") and calibrated (when it says 30%, about 30% of such patients should be positive). Logistic regression offers both: its coefficients become odds ratios, and calibration_curve checks the probabilities.

Python Heart disease: odds ratios and calibration

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The odds ratios read directly as statements: for example "each standard deviation of age multiplies the odds of disease by about 2.8, holding the other columns fixed". They describe association in this dataset, not proof that changing a habit changes the risk; see Causal Thinking. In the calibration table the observed rates track the predicted risks closely, as they should for a logistic model fit on data from the same process. A model that ranks patients well (high AUC) can still be badly calibrated, so check both, and re-check after the population changes.

More project ideas

Once these four are comfortable, try the same loop on:

  • Anomaly detection: find unusual server readings or sensor values without labels, using an isolation forest or a robust z-score, and decide how many alerts people can review; the logic of fraud capacity above applies.
  • Spam and sentiment: classify messages from text with a bag-of-words or TF-IDF pipeline, as in Text Data Basics.
  • A wine-quality score: the well-known version rates wines from 3 to 9, a mildly imbalanced ordinal target; treat it as regression first and as classification second, and compare.

For each, write a one-paragraph conclusion with the metric, the baseline it beat, the limits and the action it supports, as in the Churn project.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In the fraud project, a model that predicts 'legitimate' for everything has about 99.5% accuracy. What does that tell you?

    With a tiny positive class, the majority answer is already almost always right. Metrics on the positive class show whether fraud is actually caught.
  2. 2. Why compare approval rates by group even though the group column was not used as a model input?

    Removing a sensitive attribute does not remove the information it correlates with. Only measuring outcomes by group reveals unequal treatment.
  3. 3. What does an odds ratio of 2.0 for 'smoker' in a logistic regression mean?

    The coefficient is on the log-odds scale, so exp(coefficient) multiplies the odds. It describes association in the data, not proof of cause.
  4. 4. What does it mean that a risk model is well calibrated?

    Calibration compares predicted probabilities with observed frequencies. Ranking quality (AUC) and calibration are different properties.

Summary

  • Every project follows the same loop: frame, split, baseline, pipeline, evaluate with the right metric, interpret.
  • Wine: clean multiclass data; cross-validate and compare against a baseline.
  • Fraud: severe imbalance makes accuracy useless; use precision, recall, average precision and a capacity-based threshold.
  • Loan approval: mixed column types go through a ColumnTransformer; measure outcomes by group, because dropping a sensitive column does not make a model fair.
  • Heart disease: logistic regression gives odds ratios for explanation and probabilities that you check with a calibration curve.
  • Always finish with a written conclusion: metric, baseline, limits and the decision it supports.

Next: Project: Scrape and Analyse Product Prices.