Skip to content
Tutorials Center
My learning

Topics

Lesson 106 of 111 ยท Projects

Project: Titanic Survival

The classic first ML project, end to end: explore a passenger table, handle missing ages and cabins, engineer features, compare models with CV and read the result.

Predicting who survived the Titanic is the classic first machine learning project, because everything a real project needs fits in one small table: a binary target, numeric and categorical columns, a lot of missing values, and room for clever features. This lesson walks through the whole workflow: explore, clean, engineer features, compare models with cross-validation, and look at what the final model does.

The real passenger list is a public data set that needs a download. To keep every number reproducible in your browser, the code here generates a Titanic-like table of 891 passengers from a fixed seed, with the same kinds of columns (class, name with a title, sex, age, family members aboard, fare, cabin, port of embarkation) and similar patterns: women and children first, first class favored, and roughly 20% of ages and 75% of cabins missing. The values are made up, so do not treat them as history. When you later run the same code on the real file, it will work with the same column names.

The workflow

Explore Rates by sex, class, age; what is missing.
Clean Fill ages, handle cabin and port.
Engineer Title, family size, cabin flag.
Compare Rule, linear, forest, boosting.

Use cross-validation and a simple baseline.

1 / 2

Step 1: Explore the data

Start with the checklist from Exploratory Data Analysis: shape, missing values, the target rate, and the survival rate by each important column. Then build the engineered columns that the next step will use, and check that they make sense.

Python Titanic EDA: missing data, survival rates and engineered columns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What the table shows.

  • Missing data. 197 ages (22%), 675 cabins (76%) and 2 embarkation ports are missing. Missing is not the same everywhere, so we handle each column differently.
  • Sex and class dominate. 67% of women survived against 24% of men. Survival falls with class (53%, 40%, 32%), and the two effects combine: 86% of first-class women survived against 17% of third-class men.
  • Age matters, with a catch. Children aged 12 or under had a 73% survival rate, adults over 35 only 28%. The age band for people over 60 has just 6 people, so its 0% is meaningless. Always look at the count.
  • Cabin missing is informative. Passengers with a recorded cabin survived at 50.5%, against 35.7% without one, mostly because cabins were recorded for first class. A column full of holes can still carry signal, so a "has cabin" flag is a feature, not just something to fix.
  • The name column hides a gem. The title in the name (Mr, Mrs, Miss, Master, Dr) separates children (Master) from men, and it also gives us a way to fill missing ages: the median age is 5 for Master, 21 for Miss, 36 for Mrs and 33.5 for Mr. A title-based age is a much better guess than the overall median.

Step 2: Features, a pipeline, and honest comparison

The engineered features are:

  • title: extracted from the name with a regular expression.
  • age filled with the median age of the passenger's title, plus an age_missing flag.
  • family_size = siblings/spouses + parents/children + 1, and is_alone.
  • has_cabin (the cabin text itself has too many distinct values to use directly).
  • fare_per_person = fare divided by family size.

To avoid leakage, all fitted statistics (the median age per title) must be learned from the training folds only. So the feature steps live in a small custom transformer Engineer (Custom Transformers) at the front of a pipeline, followed by a ColumnTransformer for scaling and one-hot encoding. We compare a "basic" pipeline (median imputation, no engineered features) with the engineered one, for three model types and two simple baselines: the majority class and the rule "women survive, men do not".

Cross-validation is repeated twice (5 folds each) and we report the mean and the spread. The data is small, so the spread is as important as the mean.

Python Models and features compared with repeated cross-validation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The honest reading is humbling, and it is the most valuable thing in this lesson. The majority-class baseline scores 0.607. The one-line rule "women survive" scores 0.732, and every model lands at 0.73 to 0.76. Feature engineering helps logistic regression a little (0.735 to 0.756) and does almost nothing for the forest (0.734 to 0.750) and boosting (0.744 to 0.748). Each of those differences is smaller than the fold-to-fold spread (the +/- values are 0.02 to 0.04), so none of them is evidence that one model beats another. Most of the signal in this table is "who is female, and which class", and a model quickly learns it. A complicated model adds little because there is little more to learn. This is why you always include a simple rule as a baseline, and why you should not choose a model from a difference of one point on 891 rows.

Step 3: Evaluate the final model, and read it

We pick logistic regression with the engineered features, partly because it scored best but mostly because it is simple and readable. The example below fits it on 75% of the data, tests once on the rest, looks for groups where the model is weaker, ranks the inputs with permutation importance (shuffle one column of the test set and see how much accuracy falls, see Feature Importance) and predicts four made-up passengers.

Python Final test, group errors, importance and new passengers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

On the held-out 223 passengers, accuracy is 0.807 with a ROC AUC of 0.866. That is higher than the cross-validated 0.756, and the gap is a reminder of what a single split does: with 223 test rows, a few lucky rows move accuracy by several points. Report the cross-validated number as the estimate, and the test number as one draw. The confusion matrix shows 24 survivors predicted dead and 19 deaths predicted alive. By group, accuracy is 0.75 for women and 0.84 for men, and 0.71, 0.81 and 0.84 across the three classes: the model is weakest for first class and for women, the groups where the outcome is least predictable by simple rules.

The permutation importance ranks name first (0.115). That does not mean the name itself matters, because the model reads the title from it, which stands in for sex and age. Then come sex (0.061) and age (0.043). Features that are merely proxies for each other share the importance in unpredictable ways, so read these numbers as a rough ranking, not as exact contributions. The made-up passengers behave sensibly: a first-class woman scores 0.87, an adult man in third class 0.12, a young third-class boy 0.68 (the "Master" title and young age matter more than his class), and a second-class woman with an unknown age 0.72.

Where to go from here

  • Submit to a competition. The real data is on Kaggle, a standard place to practice. Use cross-validation scores, and do not tune for the public leaderboard.
  • Try more features: surnames shared by group (families traveling together), the ticket number, the deck letter from the cabin.
  • Tune. A small grid or random search on the forest or boosting model, judged by repeated CV.
  • Be careful about the story. Predicting a historical event is a classification exercise. In real projects, a model that uses sex or class should make you think about fairness.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. The rule 'women survive, men do not' scores 0.732 and the best model 0.756. What is the sensible conclusion?

    A simple baseline captures most of what is learnable here. A one-to-two point gain with a spread of 0.02 to 0.04 is not evidence of a better model.
  2. 2. Why is the median age per title computed inside a pipeline step that learns it in fit()?

    If the medians were computed on the whole table before splitting, the held-out rows would influence the values used for training. Learning them in fit() keeps every fold honest.
  3. 3. Passengers with a recorded cabin survived more often. How should the cabin column be used?

    Cabins were recorded mostly for first class, so 'has a cabin' is a proxy for class and wealth. A flag keeps that signal without trying to use hundreds of unique strings.
  4. 4. Permutation importance ranks 'name' above 'sex'. What is the best explanation?

    The title is extracted from the name inside the pipeline, so shuffling the name column destroys the title feature. Features that are proxies for each other make importance values hard to interpret exactly.

Summary

  • Explore first: missing values, target rate, survival by sex, class and age bands, always with counts.
  • Missing values can carry signal (has_cabin), and titles in names give both a feature and a better age fill.
  • Put feature engineering inside the pipeline (a custom transformer) so statistics are learned from training folds only.
  • Always include a simple rule baseline; here "women survive" is within two points of every model, and differences smaller than the CV spread are not evidence.
  • A single test split is one draw; report the cross-validated estimate, check errors by group, and read the model before trusting it.

Next: practice the fundamentals by hand in ML Coding Interview Questions.