Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 37 ยท Ordinary Differential Equations

Adaptive Step Size and Stiff ODEs

Let the solver choose its own step size with embedded Runge-Kutta pairs, and learn what stiffness is and why implicit methods like BDF and Radau solve it.

A fixed step size is wasteful. When the solution changes slowly you could take big steps; when it changes quickly you need small ones. Adaptive step-size control lets the algorithm estimate its own error and adjust h as it goes, so you specify the accuracy you want and not the number of steps.

The second half of this lesson covers stiff problems, where even a perfect adaptive explicit method is forced into tiny steps. Recognizing stiffness, and switching to an implicit method, is one of the most valuable practical skills in ODE solving.

Embedded Runge-Kutta pairs

How can a method estimate its own error cheaply? Compute two answers of different orders from the same stage values: a fourth-order y4 and a fifth-order y5. The difference |y5 - y4| estimates the error of the lower-order result. This is an embedded pair; the famous ones are Dormand-Prince (RK45, SciPy's default) and Bogacki-Shampine (RK23).

Control logic after each trial step of size h with estimated error err and tolerance tol:

  • If err <= tol: accept the step.
  • Choose the next step as h_new = h * safety * (tol / err)^(1/(p+1)), where p is the lower order and safety is about 0.9.
  • If err > tol: reject the step and retry with h_new.

The simplest embedded pair is Euler (order 1) inside Heun (order 2), which fits in a few lines.

Python Adaptive step size with an Euler/Heun pair

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The step sizes start small, where the exp(-20 t) transient lives, and grow as the solution becomes smooth.

Adaptive solvers in SciPy

scipy.integrate.solve_ivp does all this with rtol (relative) and atol (absolute) tolerances. The error per step is kept below atol + rtol * |y|. Typical defaults are 1e-3 and 1e-6; tighten them to 1e-8 or so for high accuracy.

Python solve_ivp with tolerances

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Tighter tolerance means more steps, but never more than needed. Note that rtol is a per step control, not a guarantee on the global error; the final error can be larger than rtol.

What is stiffness?

An ODE is stiff when it has components that decay (or change) on very different time scales, one of them very fast but already negligible, while the solution you care about evolves slowly. Explicit methods must keep h small enough for the fast mode to stay stable, even though that mode contributes nothing to the answer. The step size is then limited by stability, not accuracy.

Classic example: y' = -1000 (y - cos t). The solution quickly approaches cos t, then moves slowly, but explicit methods need h below about 0.002 the whole time.

flowchart LR
  A["Fast decaying mode<br/>(already negligible)"] --> C["Explicit solver:<br/>h limited by stability"]
  B["Slow mode<br/>(what you want)"] --> C
  C --> D["Thousands of tiny steps"]
Python Explicit versus implicit on a stiff problem

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All methods agree on the answer, but RK45 takes about 1,800 steps and 10,800 evaluations of f, while Radau needs 66 steps and 500 evaluations and BDF 159 steps and 362. For an expensive right-hand side that difference is the whole budget.

Implicit methods and how to pick one

Implicit methods solve an equation each step (Newton iteration, often with a Jacobian), which allows h to be chosen by accuracy alone for stiff problems.

  • Radau (implicit Runge-Kutta, order 5): robust for stiff problems at high accuracy.
  • BDF (backward differentiation formulas, variable order 1-5): the classic stiff solver, efficient for large systems.
  • LSODA: switches automatically between stiff and non-stiff modes.
  • RK45 / DOP853: non-stiff problems; DOP853 is best for very tight tolerances.

Giving the solver a Jacobian (jac=) speeds up implicit methods and makes them more reliable on big systems.

If RK45 is taking surprisingly many steps with a smooth-looking solution, or sol.nfev is huge, suspect stiffness and try method="Radau" or "LSODA". It is a one-word change.

A stiffness-prone example: chemical kinetics

Chemical reactions with very different rate constants are stiff. The Robertson problem is the standard test: three species, rate constants spanning nine orders of magnitude.

Python Robertson chemical kinetics

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The total a + b + c stays at 1 (mass is conserved), a handy sanity check on any chemical simulation.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How does an embedded Runge-Kutta pair estimate its error?

    The difference between the higher-order and lower-order results approximates the error of the lower-order one.
  2. 2. What limits the step size of an explicit method on a stiff problem?

    The fast mode forces h below the stability limit even though it no longer affects the answer.
  3. 3. Which SciPy solver would you try first on a suspected stiff problem?

    Implicit solvers like Radau and BDF, or the automatic LSODA, handle stiffness efficiently.
  4. 4. What does rtol control in solve_ivp?

    rtol and atol bound the estimated local error per step; the global error can be larger.

Summary

  • Adaptive solvers use embedded pairs to estimate error and adapt h; you set rtol and atol.
  • Stiff problems have widely separated time scales; explicit methods are then limited by stability, not accuracy.
  • Implicit methods (Radau, BDF, LSODA) take large steps on stiff problems at the cost of solving equations per step.
  • A huge step count with a smooth solution is the telltale sign of stiffness.

Next: Systems and higher-order ODEs