Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 40 ยท Projects and Practice

Project: House Price Regression

Predict home prices with LightGBM on a generated dataset: baselines, categorical features, a log target, early stopping, error analysis by price band and feature contributions.

House prices are the classic regression exercise: a mix of numeric and categorical features, a skewed target, interactions (a large house matters more in an expensive neighborhood) and a clear business question ("how wrong are we, and where?"). In this project you build the model step by step on generated data that has these properties, so everything runs in your browser and the true rules are known.

Step 1: the data and the baselines

The price depends on living area, age, distance to the center, number of rooms, and the neighborhood, which multiplies the whole price. The target is right-skewed because the effects are multiplicative, which is typical for prices.

Python Step 1: generate data and set baselines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Always set simple baselines first. The mean predictor shows the scale of the problem. The linear model (with one-hot neighborhoods) is stronger, but it adds effects, while the true price multiplies them, so it leaves a lot on the table. Any boosted model must beat both.

Step 2: LightGBM with a native categorical feature

LightGBM can split on a pandas category column directly, without one-hot encoding (LightGBM Dataset and Categorical Features). We also use early stopping on a validation part carved out of the training data, so the test set stays untouched.

Python Step 2: train LightGBM with early stopping

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In one run LightGBM reached an MAE of about 12,400 (RMSE 16,700 for the raw target), against about 15,200 for the linear baseline and 53,800 for the mean predictor. It beats the linear model because it learns the multiplicative neighborhood effect and the interaction between area, age and distance without being told. Second, compare the raw-price and log-price rows: here they are almost identical (MAPE 0.068 against 0.067), because the noise in this data is small and the trees can approximate the multiplicative structure either way. A log target turns multiplicative effects into additive ones and treats a 10% error the same for cheap and expensive houses, which tends to help more when prices span a wide range or the noise is larger. It also guarantees positive predictions. The trade-off is that it optimizes the squared error of log prices, not of dollars. Pick the version that matches what the business measures, and let validation decide.

Step 3: where is the model wrong?

An overall MAE hides structure. Look at the error by price band, and at the biggest misses:

Python Step 3: error analysis by price band

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In one run the mean absolute error grew from about 7,400 in the cheapest quarter to about 19,500 in the top quarter, while the percentage error stayed between 6% and 7% in every band and every neighborhood: an error of 5% costs more on a large house, but the relative accuracy is stable. The largest misses are usually unusual houses (very large and very old, or very far from the center), where the model has few similar examples. In real work you would investigate them: are they data errors or genuinely unusual properties?

Step 4: what drives the price?

Use LightGBM's built-in contributions (pred_contrib=True, the same SHAP idea as before) to see the average effect of each feature on the log price:

Python Step 4: feature contributions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Area dominates (mean absolute contribution about 0.25 log points), followed by neighborhood (0.14) and age (0.10), which matches how the data was generated. The neighborhood contributions rank the districts in the right order, and the recovered premiums against Parkview (+45% for Hilltop, -9% for Station) are very close to the multipliers built into the generator (1.45 and 0.90 against 1.00). Because the contributions are in log points, differences between them are percentage effects, an additional benefit of the log target.

Extensions

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why try a log transform of the target for prices?

    On the log scale, a 10% error is the same size for cheap and expensive houses, and effects combine by addition.
  2. 2. What is the purpose of the baselines (mean and linear regression)?

    Without baselines you cannot tell whether a score is good.
  3. 3. How does LightGBM use a pandas category column?

    LightGBM finds the best partition of categories itself, so no one-hot encoding is needed.
  4. 4. The mean absolute error grows with price while the percentage error stays flat. What does it show?

    Relative accuracy is stable; absolute dollar errors naturally grow with the price level.

Summary

  • Start with baselines, then train LightGBM with a held-out validation set for early stopping and an untouched test set.
  • Use native categorical features, and consider a log target for skewed, multiplicative prices.
  • Analyze errors by price band, neighborhood and the largest misses, not only the average.
  • Contributions (pred_contrib=True) show the average effect of each feature, in log points for a log target.

Next: Project: Sales Forecasting with Lag Features