Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 68 ยท Differential Equations

Solving ODEs Numerically

Build Euler and Runge-Kutta ODE solvers from scratch, see how step size, accuracy and stability interact, and use scipy solve_ivp for systems of equations.

Only a small fraction of real differential equations have tidy formulas for their solutions. A pendulum swinging through a large angle, a predator-prey ecosystem, a rocket with changing mass and a chemical reactor with temperature feedback all need a different approach: compute the solution step by step on a computer. A numerical ODE solver starts at the initial condition and repeatedly uses the equation itself to move a short time forward.

In this lesson you will write the two most important solvers, Euler's method and the fourth-order Runge-Kutta method (RK4), see how step size controls the error, discover why some equations make a solver blow up, and then switch to the professional tool scipy.integrate.solve_ivp, including for systems of equations. It builds directly on derivatives, linearization and the models from first-order ODEs.

Euler's method

Take the initial value problem y' = f(t, y), y(t0) = y0. The derivative tells you the slope at the current point, so follow the tangent line for a short step h:

y_next = y + h*f(t, y), t_next = t + h.

This is the tangent-line approximation applied repeatedly, and it traces a polygon that hugs the true solution curve. Every step makes a small error because the real slope changes during the step. Those errors accumulate. For Euler's method the total error at a fixed final time shrinks in proportion to h: halve the step and you roughly halve the error. We say Euler is first order.

Runge-Kutta: more slopes per step

The error comes from using only the slope at the start of the step. Better methods sample the slope at several places and average. The classic RK4 uses four:

  • k1 = f(t, y): slope at the start,
  • k2 = f(t + h/2, y + h/2*k1): slope at the midpoint using k1,
  • k3 = f(t + h/2, y + h/2*k2): slope at the midpoint again, using k2,
  • k4 = f(t + h, y + h*k3): slope at the end,

then y_next = y + h/6 * (k1 + 2*k2 + 2*k3 + k4). The weights 1, 2, 2, 1 are those of Simpson's rule (see numerical integration). RK4 is fourth order: halving h divides the error by about 16. It costs four function evaluations per step, but it can use much larger steps for the same accuracy.

flowchart LR
    A["y at t"] --> B["k1: slope at start"]
    B --> C["k2: midpoint slope"]
    C --> D["k3: refined midpoint slope"]
    D --> E["k4: slope at end"]
    E --> F["weighted average gives y at t + h"]

RK4 takes four slope samples inside each step and blends them.

The code below works for a single equation and for vectors alike, because NumPy arrays support the same arithmetic. It solves y' = -2y, y(0) = 1, whose exact solution is exp(-2t), and measures the error at t = 2 for several step sizes.

Python Euler and RK4 from scratch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The error ratios when h is halved approach 2 for Euler and 16 for RK4, confirming first and fourth order. At h = 0.03125 RK4's error is about 1e-8, while Euler's is still about 2e-3. The formula error ~ C*h**p is the practical way to know your solution is converging: run at h and h/2 and compare.

Stability: when a smaller step is not optional

Accuracy is not the only concern. Consider y' = -50*y. The true solution decays to zero almost instantly. One Euler step multiplies y by 1 - 50*h. If h = 0.05 that factor is -1.5: the numerical solution flips sign and grows by 50% every step, while the real solution vanishes. For h = 0.04 the factor is exactly -1, so it oscillates forever, and only for h < 0.04 does it decay (monotonically below h = 0.02).

This is numerical instability, and it is not a bug in your code: the method itself has a stability region. Equations with very fast decaying components alongside slow ones are called stiff, and they force explicit methods like Euler or RK4 to take tiny steps. For stiff problems use implicit solvers such as "Radau" or "BDF" in solve_ivp.

Python Stable and unstable step sizes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Systems of ODEs

A higher-order equation or several interacting quantities become a system y' = f(t, y) where y is a vector. The standard trick turns any n-th order ODE into n first-order ones. For a pendulum, theta'' = -(g/L)*sin(theta), define y1 = theta and y2 = theta'; then y1' = y2 and y2' = -(g/L)*sin(y1). The same code from above handles it because f simply returns an array. The state (y1, y2) can be plotted as a curve in the phase plane, which shows motion at a glance: a closed loop is a periodic oscillation.

scipy.integrate.solve_ivp

In practice you rarely write your own solver. solve_ivp(f, (t0, t_end), y0, ...) picks step sizes automatically to meet error tolerances rtol and atol, and the default method "RK45" is an embedded Runge-Kutta pair that estimates its own error. Useful options:

  • t_eval=... returns the solution at specific times,
  • method="LSODA" or "Radau" for stiff problems,
  • dense_output=True gives a continuous interpolated solution,
  • events=... stops the integration when a condition is met (such as hitting the ground).

The function must have the signature f(t, y) and return the derivative vector. Below, a predator-prey model (Lotka-Volterra) and a large-angle pendulum are solved with it. For the pendulum with energy conserved, a good solver's phase curve should close on itself.

Python solve_ivp on a predator-prey system and a pendulum

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The populations rise and fall in lagging cycles, and the phase curve is a closed loop. The tiny energy drift in the pendulum shows the tolerance doing its job; loosen rtol to 1e-3 and watch the drift grow.

Always change the tolerance or step size and see whether the answer moves in the digits you care about. If it does not, the numerics are not what limits you. If it does, you have found out something that the plot alone would not have told you.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Euler's method is first order. If you halve the step size, the global error is roughly:

    A first-order method has error proportional to h, so halving h halves the error. RK4 would divide it by about 16.
  2. 2. How many function evaluations does one RK4 step use?

    RK4 evaluates the slope function four times (k1 to k4) per step.
  3. 3. For y' = -100y, Euler with h = 0.03 gives the factor 1 - 100h = -2 per step. What happens?

    A factor of magnitude greater than 1 means each step amplifies the value: the method is unstable at that step size even though the true solution decays.
  4. 4. How do you write the second-order ODE x'' = -4x as a first-order system?

    Introduce the velocity as a new variable: u' = v and v' = x'' = -4u.
  5. 5. Which solve_ivp method choice suits a stiff problem?

    Implicit methods such as Radau and BDF remain stable at step sizes where explicit methods like RK45 would be forced to crawl.

Summary

  • Numerical solvers step from the initial condition using the slope function f(t, y).
  • Euler's method is first order (error ~ h); RK4 is fourth order (error ~ h**4) at four slope evaluations per step.
  • Accuracy and stability are different things: too large a step makes explicit methods blow up on fast-decaying (stiff) problems.
  • Higher-order equations become first-order systems by adding derivatives as extra variables.
  • solve_ivp chooses steps adaptively from rtol/atol; use t_eval, events and stiff methods when needed.

Next: The Laplace Transform.