Skip to content
Tutorials Center
My learning

Topics

Lesson 91 of 111 ยท Projects

Project: Predicting House Prices

Build a complete regression project: explore synthetic housing data, make a preprocessing pipeline, compare linear, regularized and boosted models and analyze the errors.

In this project you will predict house prices from a handful of features, the classic regression exercise. The point is not the prediction itself but the process: look at the data before modeling, set a baseline, build one pipeline that handles messy columns, compare several models fairly with cross-validation, and finally study where the best model is wrong. That last step is where most real improvements come from.

We generate the data ourselves, so it runs instantly in your browser and no file is needed. Because we wrote the recipe, we also know the true rules behind the prices (location matters, age lowers value, size scales the price), which lets you judge whether the models found them. Every step reuses ideas from earlier lessons: Pipelines, ColumnTransformer, Regularization, Histogram Gradient Boosting and Cross-Validation.

The project plan

Explore Look at distributions and relationships first.
Baseline Know what a trivial guess scores.
Pipeline Impute, scale and encode in one object.
Compare Cross-validate several models fairly.
1 / 2

Step 1: the data and a first look

Each row is a house with six features: living area in square feet, number of bedrooms, age in years, distance to the city center in kilometers, whether it has a garage, and its neighborhood (a category). The target is the sale price. Some ages and distances are missing, as they are in real listings.

The generating rule multiplies several effects together (a price is not a sum of independent dollar amounts), and adds random noise. We keep it in a function so every later step can recreate exactly the same table from the same seed.

Python Create the data and explore it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Three observations steer the modeling:

  • Prices are skewed to the right. A few expensive houses stretch the distribution. On a log scale it is much more symmetric, and because effects multiply, a model of log(price) will fit more naturally than one of raw dollars.
  • The price-area relationship fans out: bigger houses vary more in absolute dollars. Errors measured in dollars will therefore be larger for big houses, so we should also look at percentage errors.
  • Two columns have gaps. We must impute them inside the pipeline, never on the whole table first (see Handling Missing Data).

In this lesson neighborhood and the numeric columns are always the only inputs. We never use anything derived from price, a classic leak (see Common Machine Learning Mistakes).

Step 2: split, baseline and a first model

Before touching any model, set aside a test set (20%) that stays untouched until the end. All model comparison happens on the training 80% with cross-validation. Then measure a baseline: always predicting the training median. Any real model must beat it clearly.

The preprocessing is one ColumnTransformer: numeric columns are median-imputed and scaled, the neighborhood is one-hot encoded. A plain LinearRegression predicting log(price) (via TransformedTargetRegressor, which logs the target for training and converts predictions back to dollars) gives the first real number.

Python Baseline and a first linear model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The baseline misses by about a third of the price on average (MAPE 35%). A plain linear model on the log scale cuts that to about 11%. This works so well because the price effects multiply, and a log turns multiplication into addition, which is exactly what a linear model handles. The next step asks whether more flexible models can do better.

Step 3: compare several models fairly

All candidates use the same preprocessing, the same target transform and the same cross-validation folds, so differences come from the model alone. We try a Ridge (a regularized linear model), a random forest and a histogram gradient boosting regressor. Both tree models can capture curves and interactions without us creating features by hand.

We report three metrics per model: MAE (average dollar error), MAPE (average percentage error) and Rยฒ. One number rarely tells the whole story.

Python Cross-validated comparison of four models

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The surprise is that the simple ridge model is as good as, or slightly better than, the two tree ensembles (MAE about 17,700 against 18,900 and 19,200), and the gaps are only one to two times the fold-to-fold spread in the last column. A difference that small is weak evidence either way. The explanation is the target transform: after taking logs, the structure is nearly additive and smooth, which a linear model captures with a handful of parameters, while trees must approximate smooth curves with steps and need more data to do it. Choosing the right representation beat choosing a fancier model. On real data the ranking could easily differ, which is why you compare instead of assuming. We continue with the ridge model, the simplest of the leaders; in a real project you might tune it (see Hyperparameter Tuning) while keeping the test set out of it.

Notice what we did not do: we did not try dozens of models and keep the winner of the test score. We compared on cross-validation folds, chose one model, and will now spend the test set exactly once.

Step 4: final test and error analysis

We refit the chosen model on the full training set and score it on the held-out test set. Then comes the more interesting part. A single MAE tells you how big the errors are but not where they are. We break the errors down by neighborhood and by size, list the five worst predictions, and plot predicted against actual price.

Python Test score and error analysis

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check three things in the output. First, is the test score close to the cross-validation score from Step 3? A large gap would signal leakage or overfitting through repeated peeking. Second, do the errors look the same for every neighborhood and size band? A model that is fine on average but much worse for, say, the cheapest or the largest houses may be unusable for those customers, which echoes the group checks in Fairness, Bias and Responsible ML. Third, in the percentage-error histogram, are the errors centered on zero and roughly symmetric? A shifted center is a bias you can often fix.

The remaining error is partly the noise we put into the generator (about 12% random variation in the price), which no model can predict. Knowing the noise floor is a luxury of synthetic data. On real data you estimate it by seeing where more features and more model capacity stop helping.

Ideas for taking it further

  • Add features: price per square foot of the neighborhood, age squared, a "newly built" flag, or an interaction between sqft and neighborhood (see Feature Creation and Transformation).
  • Tune the learning rate, depth and number of iterations of the boosting model with RandomizedSearchCV.
  • Compare loss functions: predicting the median with loss="absolute_error" is more robust to the odd expensive outlier.
  • Build prediction intervals (for example with quantile regression) so buyers see a range instead of a single number.
  • Check calibration of the errors over time if prices drift, and plan retraining (see Saving Models and Deploying).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why model log(price) instead of price in this project?

    Effects such as neighborhood, age and size scale the price rather than add dollars, and a log turns multiplication into addition while taming the skew.
  2. 2. Where should the missing-value imputer be fitted?

    Fitting an imputer on all rows lets information from validation or test rows influence training. Inside a Pipeline it is refitted on each training fold.
  3. 3. Four models differ in mean cross-validated MAE by less than the fold-to-fold spread. What can you conclude?

    Differences inside the spread are within noise. Prefer the simpler or cheaper model, or gather more evidence. Do not spend the test set on the choice.
  4. 4. What is the purpose of the baseline in Step 2?

    A baseline such as the median price gives the improvement a meaningful reference. A model that barely beats it adds little value.
  5. 5. Error analysis shows large houses have much bigger percentage errors than small ones. What does that suggest?

    An average error can hide poor performance on a segment. Slicing errors by group shows where the model needs more data, features or a different approach.

Summary

  • Explore first: skewed targets, fan-shaped relationships and missing values shaped our choices.
  • Lock away a test set, set a baseline, and compare models by cross-validation on the training data with identical preprocessing and folds.
  • Put imputing, scaling and encoding inside a single ColumnTransformer and pipeline; use TransformedTargetRegressor to model log(price).
  • Read differences relative to the fold-to-fold spread, and report MAE, MAPE and Rยฒ together.
  • Spend the test set once, then analyze errors by segment, biggest misses and bias to decide what to improve next.

Next up: switch from regression to a classification project with money on the line in Project: Customer Churn Classifier.