Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 39 ยท Numerical Methods

Stiff Equations and Stability

Learn what makes an ODE stiff, why explicit methods blow up on the test equation, how implicit methods such as backward Euler stay stable, and how to pick a solver.

Sometimes an adaptive solver that works beautifully on everything else grinds to a halt, taking thousands of tiny steps for a solution that looks smooth and boring. The problem is not accuracy. The problem is stability: the method would blow up unless the step is kept tiny. Such problems are called stiff.

Stiffness shows up in chemistry (fast and slow reactions together), electrical circuits, control systems and discretized PDEs. Recognizing it and choosing an implicit method can turn hours of computation into milliseconds.

The test equation

To understand stability, study the simplest decaying problem: y' = ฮป y with ฮป < 0 (or complex with negative real part), whose exact solution decays to zero. A good method should also decay.

Explicit Euler: y_{k+1} = (1 + hฮป) y_k. The numbers decay only when |1 + hฮป| < 1, i.e. h < 2/|ฮป| for real ฮป. With ฮป = -1000 you need h < 0.002. For h = 0.01, the factor is 1 - 10 = -9: the numerical solution oscillates and grows by 9 every step, while the true solution has long since vanished.

Backward Euler: y_{k+1} = y_k + h ฮป y_{k+1}, so y_{k+1} = y_k / (1 - hฮป). For ฮป < 0 the factor 1/(1 - hฮป) is always between 0 and 1, whatever h is. The method is A-stable: stable on the whole left half-plane.

Python Explicit versus implicit Euler on y' = -1000 y

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The explicit column explodes by a factor of -9 each step, the implicit one decays smoothly. Backward Euler is only first-order accurate, but its stability lets it take steps as big as the accuracy demands, not as small as stability demands.

What makes a problem stiff

A system is stiff when it contains widely separated time scales and you only care about the slow one. After a short transient the fast components have decayed, but an explicit method must still use steps small enough to resolve their stability limit forever.

For u' = A u, a handy measure is the stiffness ratio: the largest |Re ฮป| divided by the smallest. Ratios of 10^3 or more are stiff.

Python Stiffness ratio of a linear system

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Implicit methods and the cost of each step

An implicit method gives an equation for the new value. For a linear problem u' = A u, backward Euler solves (I - hA) u_{k+1} = u_k. For nonlinear problems you need Newton's method inside every step, which uses the Jacobian matrix of f. A step is more expensive, but the number of steps drops hugely for stiff problems.

Python Backward Euler with a linear solve

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With only 50 steps of h = 0.1, the fast component is damped to nothing, as it should be, and the slow component has the right size, though it is off by about 25% because backward Euler is only first order and the step is coarse. Explicit Euler with that step would have blown up: each step multiplies the fast mode by 1 - 100 = -99.

Choosing a solver in SciPy

solve_ivp offers several implicit methods:

Method Type Use it for
Radau implicit RK, order 5 stiff problems, high accuracy
BDF variable-order multistep stiff problems, large systems
LSODA switches automatically when you do not know
RK45, DOP853 explicit non-stiff problems

If a non-stiff method takes a huge number of steps for a smooth-looking solution, switch to Radau or LSODA and compare. Supplying the Jacobian with jac= speeds up implicit solvers and improves their robustness.

Python Robertson chemical kinetics: explicit versus implicit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All methods agree on the answer, which also shows the conserved total a + b + c = 1, but the explicit method needs far more steps. The longer the interval, the bigger the difference: the explicit step count grows in proportion to the time span, while the implicit solvers can take ever larger steps once the fast reaction has settled.

Is my problem stiff?

Tiny steps Smooth solution, yet millions of steps.
Huge eigenvalue ratio Fast and slow modes together.
Fast decay Transients vanish quickly.
Fix Switch to Radau, BDF or LSODA.

Stiffness is about stability, not accuracy.

1 / 2

Stability regions

Every method has a stability region: the set of z = hฮป in the complex plane for which it decays on the test equation. Explicit Euler's region is the disk of radius 1 around -1. RK4's is somewhat larger, reaching about -2.78 on the real axis. Implicit methods can include the entire left half-plane (A-stable). Two practical consequences:

  • Making an explicit method higher order does not remove the stability limit on h; it only changes its size a little.
  • Oscillatory problems with eigenvalues near the imaginary axis stress methods differently: explicit Euler is unstable for pure oscillation at any step, as you saw with the harmonic oscillator.

Common mistakes

  • Blaming accuracy when the real issue is stability, and trying ever tighter tolerances with an explicit method.
  • Using a stiff solver for everything. For non-stiff problems explicit methods are faster per step.
  • Judging stiffness from the solution's look rather than from the eigenvalues of the Jacobian.
  • Forgetting that implicit methods need the Jacobian, which solve_ivp otherwise approximates numerically.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. For y' = -500 y, explicit Euler is stable only if h is less than:

    The condition |1 - 500h| < 1 gives h < 2/500 = 0.004.
  2. 2. Backward Euler on y' = ฮปy with ฮป < 0 is stable for:

    The factor 1/(1 - hฮป) lies strictly between 0 and 1 for every h > 0, so the method is A-stable.
  3. 3. A stiff problem typically has:

    Fast decaying modes force tiny explicit steps even when the interesting solution is slow.
  4. 4. Which solve_ivp method is a sensible default for a stiff problem?

    Radau (or BDF, or LSODA) is implicit and handles stiffness.

Summary

  • Stiffness is a stability problem: fast decaying modes force tiny explicit steps.
  • The test equation y' = ฮปy shows explicit Euler needs h < 2/|ฮป|; backward Euler has no such limit.
  • Implicit methods solve an equation per step, using the Jacobian.
  • In SciPy use Radau, BDF or LSODA for stiff problems; compare the number of steps with RK45.
  • The stiffness ratio of the Jacobian's eigenvalues is a quick diagnostic.

Next: Boundary Value Problems and Shooting