Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 28 ยท Linear Regression in Depth

Formulas and Patsy

Write regression models as formulas like y ~ x1 + np.log(x2) in Statsmodels. Learn the Patsy operators, transformations, the no-intercept form and how to inspect design matrices.

Building a design matrix by hand is tedious: add the constant, log-transform one column, square another, turn text categories into indicator columns. The formula API does all of this from a short string. It uses a library called Patsy, which copies the model syntax of the R language.

Python
import statsmodels.formula.api as smf
res = smf.ols("price ~ area + age", data=df).fit()

The left side of ~ is the outcome, the right side lists the predictors, and data= is the DataFrame whose column names the formula uses. The intercept is added automatically, which is a big difference from sm.OLS.

Python Your first formula

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The operators

Formula piece Meaning
y ~ x1 + x2 Both predictors, plus an intercept
y ~ x1 - 1 or y ~ 0 + x1 Remove the intercept
y ~ x1 : x2 Only the interaction (product) of the two
y ~ x1 * x2 x1 + x2 + x1:x2 (main effects and interaction)
y ~ C(g) Treat g as a categorical variable
y ~ I(x**2) Compute x**2 literally (needed because ** has a formula meaning)
y ~ np.log(x) Call any function that is in scope

Interactions and categories get their own lesson: Categorical Variables and Interactions. Here we focus on transformations.

Transformations inside the formula

Any NumPy function works, and so does your own. The transformation happens when the model is built, and it is also replayed automatically at prediction time. That is a major advantage: you cannot forget to log-transform the new data.

Python Log and polynomial terms in a formula

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The log model wins because the data were generated with a logarithmic curve. Comparing AIC across formulas is a handy way to test functional forms. Remember that a model that is linear in its coefficients can still draw a curve: I(income**2) is just another column.

Write I(x**2), not x**2. Inside a formula ** is a Patsy operator for interactions, so a bare x**2 quietly gives you just x with no error.

Looking at the design matrix

When a formula behaves oddly, look at what Patsy built. dmatrices returns the outcome and the design matrix, and res.model.exog_names lists the column names.

Python Inspect what Patsy builds

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Special tools

  • Standardizing: y ~ scale(x1) + scale(x2) centers and scales within the formula, making coefficients comparable.
  • Splines: bs(x, df=4) builds B-spline basis columns for flexible curves; cr(x, df=4) uses cubic regression splines.
  • Column names with spaces or symbols: wrap them with Q("my col"), for example y ~ Q("floor area").
  • Subsetting: pass subset=df.age > 20 to smf.ols(...).
  • Missing values: rows with NaN in any used column are dropped by default (missing="drop" in Patsy's formula mode). Check res.nobs.
Python Q(), scale() and splines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The spline follows the sine wave a straight line cannot capture.

Formulas versus arrays: when to use which

Use formulas when you work with DataFrames, categories or transformations. Use the array API when you already have a numeric matrix, for example from NumPy code or a feature pipeline. The model classes are the same underneath: smf.ols simply calls sm.OLS with a matrix Patsy made.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How does the intercept behave in smf.ols('y ~ x', df)?

    Formulas add an intercept by default. Remove it with '- 1' or '0 +'.
  2. 2. How do you add a squared term safely?

    I() tells Patsy to evaluate the Python expression as written. A bare ** is a Patsy operator.
  3. 3. What does y ~ a * b expand to?

    The star gives both main effects and their interaction.
  4. 4. What is a benefit of putting np.log(x) inside the formula?

    Patsy stores the recipe, so predict() rebuilds the matrix from raw columns.

Summary

  • smf.ols("y ~ x1 + x2", data=df) builds the design matrix from a formula and adds an intercept.
  • Use - 1 to drop the intercept, * and : for interactions, C() for categories, I() for literal arithmetic.
  • Functions like np.log, scale, bs work inside formulas and are replayed in predict.
  • patsy.dmatrices shows the exact matrix when debugging.

Next: Categorical Variables and Interactions