Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 28 ยท Other Models

LOWESS and KDE

Use nonparametric tools in Statsmodels: LOWESS to draw a flexible trend line without a formula, and kernel density estimation to estimate a distribution shape.

Parametric models such as OLS commit to a formula: a line, a parabola, a logarithm. When you do not know the shape, or you want a first look at the data before choosing a model, nonparametric tools let the data speak. Statsmodels includes two favorites in statsmodels.nonparametric: LOWESS for smoothing a relationship and KDE for smoothing a distribution.

LOWESS: locally weighted scatterplot smoothing

LOWESS fits many small weighted regressions. To estimate the curve at one x, it takes the nearest fraction frac of the points, gives close points more weight, fits a straight line and reads off the fitted value. Repeating this for each x gives a smooth curve with no global formula.

The single important choice is frac: small values follow the data closely (wiggly, noisy), large values give a smooth, stiff curve.

Python LOWESS with different bandwidths

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

frac=0.1 chases noise, frac=0.8 flattens the sine wave, and 0.3 is a sensible middle. There is no standard-error output, so LOWESS is a descriptive tool, ideal for scatterplot trend lines and for spotting non-linearity before modeling.

Using LOWESS to check a linear model

A classic diagnostic: smooth the residuals against a predictor or fitted values. A flat line near zero is healthy; a curve means missing structure (Residual Diagnostics).

Python LOWESS of residuals reveals curvature

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The smoothed residuals of the straight-line model swing widely (the curve left a pattern); after adding the squared term they stay near zero.

KDE: kernel density estimation

A histogram depends on bin edges and bin width. A kernel density estimate replaces each observation with a small smooth bump (the kernel, usually Gaussian) and adds the bumps up, giving a smooth estimate of the underlying distribution. The key setting is the bandwidth: the width of each bump.

sm.nonparametric.KDEUnivariate is the main class. Call fit(bw=...), then read support (the x grid) and density.

Python KDE of a two-humped distribution

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The curve shows the two humps clearly. A too-narrow bandwidth (bw=0.2) creates spurious bumps; too wide hides real structure. Rules such as Scott's and Silverman's pick sensible defaults, and bw="cv_ml" chooses it by cross-validation.

Other nonparametric pieces

  • KernelReg fits a nonparametric regression of y on x with kernel weights, giving a smooth curve with a mean function and marginal effects.
  • KDEMultivariate estimates joint densities of several variables.
  • Compare shapes of two samples visually by overlaying their KDEs; for a formal comparison, use tests such as Kolmogorov-Smirnov in SciPy.
Python KernelReg: nonparametric regression

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Nonparametric methods need more data than parametric ones to reach the same precision, and they cannot extrapolate beyond the observed x range. Use them to explore, then decide whether a simple parametric model is good enough.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the frac argument of lowess control?

    Small frac follows the data closely; large frac gives a smoother, stiffer curve.
  2. 2. What is a good use of LOWESS on regression residuals?

    A smoothed residual curve that departs from zero reveals missing structure.
  3. 3. In kernel density estimation, what does too small a bandwidth cause?

    Narrow kernels follow each observation, creating noise; wide kernels oversmooth.
  4. 4. Which statement about nonparametric methods is true?

    They are flexible but data hungry, cannot extrapolate and depend on a smoothing parameter.

Summary

  • LOWESS draws a flexible trend line; frac sets smoothness. It is descriptive, with no standard errors.
  • Smooth residuals to spot curvature after fitting a parametric model.
  • KDE estimates a distribution; bandwidth trades noise against oversmoothing.
  • KernelReg and KDEMultivariate extend these ideas; nonparametric tools cannot extrapolate.

Next: Pitfalls and Debugging