Skip to content
Tutorials Center
My learning

Topics

Lesson 2 of 28 ยท First Steps

Your First OLS Model

Fit ordinary least squares in Statsmodels with arrays and DataFrames, read the coefficients, make predictions and plot the fitted line, step by step.

Ordinary least squares (OLS) is the workhorse of statistics. It draws the straight line, or in higher dimensions the flat surface, that makes the sum of squared vertical gaps between the data and the line as small as possible. In this lesson you fit your first real OLS model, read the key numbers and use it to predict.

The model is y = b0 + b1*x1 + b2*x2 + ... + error. The b values are coefficients: b1 is the average change in y when x1 rises by one unit and everything else stays fixed.

The design matrix

Statsmodels' array API wants X, a table with one column per predictor. The intercept b0 multiplies a column of ones, and you must add that column yourself with sm.add_constant. Forgetting it forces the line through the origin, a very common mistake (see Pitfalls and Debugging).

Python OLS with two predictors

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because X is a DataFrame, the parameters carry the names const, area and age. With plain NumPy arrays you would get const, x1, x2.

Reading the main attributes

The results object is a toolbox. These attributes are the ones you use daily:

Attribute Meaning
res.params Estimated coefficients
res.bse Standard errors of the coefficients
res.tvalues, res.pvalues t statistics and p-values
res.conf_int() Confidence intervals (95% by default)
res.rsquared, res.rsquared_adj Share of variance explained
res.resid Residuals: actual minus fitted
res.fittedvalues Model predictions for the training data
Python Inspect the results object

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that the residual mean is essentially zero. When an intercept is included, OLS forces the residuals to average out to zero. That is one reason the intercept matters.

Predicting new data

res.predict(new_X) applies the fitted equation. The new table must have the same columns in the same order, including the constant.

Python Predict and get prediction intervals

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two intervals appear. The mean interval says where the average price of all such houses lies. The wider observation interval says where a single new house's price probably falls. Use the second when you predict individual cases.

Plotting the fit

For one predictor, a plot shows the line against the data. In the lesson editors a figure appears below the code.

Python Scatter plot with the fitted line

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the coefficient of a predictor mean in multiple OLS?

    Coefficients are partial effects: each one is adjusted for the other predictors in the model.
  2. 2. You forget sm.add_constant. What happens?

    The array API fits exactly the columns you pass. Without a column of ones there is no intercept.
  3. 3. Which interval is wider and suits a single new observation?

    A single outcome includes the random error as well as the uncertainty of the average, so its interval is wider.
  4. 4. Where do you find the residuals?

    res.resid holds actual minus fitted values.

Summary

  • OLS minimizes the sum of squared residuals.
  • Build X with sm.add_constant, fit, then read params, bse, pvalues and conf_int().
  • Predict with res.predict(new_X) using the same columns as training.
  • get_prediction(...).summary_frame() gives both mean and observation intervals.

Next: Reading the Summary Table