Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 38 ยท Multirate and Adaptive Filters

The LMS Adaptive Filter

Build the least-mean-squares adaptive filter step by step, learn how the step size controls convergence and noise, and use it to identify an unknown system.

All filters so far had fixed coefficients that you computed in advance. An adaptive filter adjusts its own coefficients while it runs, driven by an error signal. That makes it useful when the right filter is unknown or changes: an echo canceller in a phone, a noise canceller in headphones, an equalizer for a drifting channel. The simplest and most widely used algorithm is LMS (least mean squares), invented by Widrow and Hoff.

The setup

An adaptive FIR filter has weights w of length L. At every time step:

  1. Form the input vector u[n] = [x[n], x[n-1], ..., x[n-L+1]].
  2. Compute the output y[n] = w . u[n].
  3. Compare with the desired signal d[n] to get the error e[n] = d[n] - y[n].
  4. Update the weights: w <- w + mu * e[n] * u[n].

mu is the step size. The update is a stochastic gradient descent on the mean squared error: each step nudges w in the direction that would have made e[n] smaller for this one sample.

The adaptive loop

Input x Fed through the filter
Desired d What we want to match
Error e d minus the filter output
Update w grows by mu * e * input

No signal statistics needed, just one multiply-add per tap.

1 / 2

System identification

The classic demonstration: an unknown FIR system h produces d = h * x. The adaptive filter sees the same input and the output, and its weights should converge to h.

Python LMS identifies an unknown filter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The learned weights match the true ones to several decimals, and the error falls from order 1 to nearly zero. With no noise in d, LMS converges all the way.

The step size

mu controls everything:

  • Too small: converges slowly; the filter cannot track changes.
  • Larger: faster convergence and tracking, but more noisy fluctuation of the weights (excess error, called misadjustment).
  • Too large: diverges. A safe upper bound is mu < 2 / (L * sigma_x^2), with sigma_x^2 the input power; in practice use a fraction of this, such as one tenth to one fifth.
Python Effect of the step size

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With the noise variance 0.01 the ideal MSE is 0.01. Small mu is slow but reaches close to 0.01; bigger mu settles sooner but sits higher.

Normalized LMS

LMS is sensitive to the input power: if the input gets louder, the effective step size grows. NLMS divides by the input energy:

w <- w + mu * e[n] * u[n] / (eps + u . u)

Now 0 < mu < 2 works regardless of the input scale. It is the standard practical version.

Python NLMS handles loud and quiet inputs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For the quiet input, plain LMS with a fixed mu has barely moved (slow); for the loud input it can blow up. NLMS gets a good result at every scale.

Convergence speed and eigenvalue spread

LMS converges at a rate set by the eigenvalue spread of the input correlation matrix, the ratio between the strongest and weakest modes. White noise has spread 1 and converges fastest; highly colored input (like speech or low-pass noise) has a large spread and some directions converge very slowly. Remedies include the affine projection algorithm, transform-domain LMS and RLS (recursive least squares), which converges faster for more computation per sample.

Applications

  • Echo cancellation: model the echo path from loudspeaker to microphone; subtract the estimate.
  • Channel equalization: undo the distortion of a communication channel.
  • Prediction: use past samples to predict the next one (linear predictive coding).
  • Noise cancellation: next lesson, adaptive noise cancellation.

LMS filters are not stable by default: a bad step size, quantization or a missing leakage term can make weights drift. Production code often adds a small leakage factor w <- (1 - mu*gamma) w + ... and clamps the step size.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the LMS weight update?

    Each weight moves by the step size times the error times the corresponding input sample, a stochastic gradient step.
  2. 2. What does a larger step size do?

    Large steps track quickly but jitter around the optimum, and too large a step makes the algorithm diverge.
  3. 3. Why use NLMS instead of LMS?

    Dividing by u.u removes the dependence on input power, making the stable range of mu independent of signal level.
  4. 4. In system identification with LMS, what is the desired signal d?

    The adaptive filter tries to reproduce the unknown system's output from the same input, so its weights approach the system's impulse response.

Summary

  • An adaptive filter updates its weights from an error signal; LMS uses w += mu * e * u.
  • System identification shows convergence to the unknown impulse response.
  • The step size trades speed against noise and has a stability bound; NLMS normalizes it.
  • Colored inputs slow convergence; RLS and other algorithms trade computation for speed.

Next: Adaptive Noise Cancellation