Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 28 ยท Projects

Project: House Price Model

Build a complete regression workflow in Statsmodels: explore the data, fit and improve an OLS model, run diagnostics, use robust errors and validate predictions on a holdout.

In this project you play the role of an analyst asked: what drives house prices in our city, and how accurately can we price a new listing? You will use almost everything from the first four chapters: formulas, categories, transformations, diagnostics, robust errors and a holdout test.

The data are simulated with a known recipe (so you can check your answers): price grows with area, falls with building age, depends on the neighborhood, and the noise is larger for expensive homes, which makes a log model natural.

flowchart LR
    A["Explore"] --> B["Baseline OLS"]
    B --> C["Diagnostics"]
    C --> D["Improve: logs, categories"]
    D --> E["Robust errors"]
    E --> F["Holdout test"]
    F --> G["Report"]

Each step feeds the next; diagnostics send you back to improve the model.

Step 1: build and explore the data

Python Create and explore the data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look for skew, ranges and suspicious values before modeling. Prices are right-skewed, as money usually is.

Step 2: a baseline model

Python Baseline linear model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The baseline already explains most variance, and rooms adds little once area is known (it overlaps with area). But does the model meet its assumptions?

Step 3: diagnostics

Python Check assumptions of the baseline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The residual spread is clearly larger for expensive homes (heteroscedasticity) and the residuals are skewed. This is the footprint of multiplicative effects: each factor scales the price. Time to use logs (Residual Diagnostics).

Step 4: improve with logs

Taking log(price) turns multiplication into addition and often cures both problems. Coefficients become approximate percentage changes. We also test whether the log of area is better than area itself.

Python Log model and comparison

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The log-log model fits best, and the heteroscedasticity disappears. The coefficient of log(area) is close to 1: a 1% larger home costs about 1% more. The age coefficient says each extra year lowers the price by roughly 0.8%, matching the recipe.

Step 5: robust errors and validation

For the final report we use robust standard errors (cheap insurance) and test on 150 homes that the model never saw. Predictions come back on the log scale, so we convert with np.exp (a small correction is possible, but the effect is minor for small residual variance).

Python Holdout test and the final answer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The typical error is about 8-10%, consistent with the 12% noise we built in. The interval communicates the uncertainty to the client: a price estimate is a range, not a point.

What to write in the report

  1. Findings in plain language: neighborhood, area and age effects as percentages, with confidence intervals.
  2. Model quality: holdout error as a percentage, not just R-squared.
  3. Assumptions: log scale chosen after residual checks; robust standard errors used.
  4. Limits: only the observed ranges (35-220 m2, age up to 60); association, not causation; no information on renovation quality or exact location.

Extensions to try

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. Why was log(price) a better outcome than price here?

    Multiplicative structure and heteroscedasticity are typical signs that a log transform is appropriate.
  2. 2. In a log-log model, what does the coefficient of log(area) represent?

    With both sides in logs, the slope is an elasticity.
  3. 3. Why evaluate on 150 homes not used in fitting?

    Out-of-sample error is an honest measure of how the model will perform in use.

Summary

  • Explore, fit a baseline, run diagnostics, improve, validate and report: the loop of a real regression project.
  • Heteroscedastic, skewed money data usually call for a log transform.
  • Report holdout error and prediction intervals, plus the limits of the model.

Next: Project: Customer Churn