Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 47 ยท Numerical Foundations

Derivatives and Integrals on a Grid

Learn finite-difference derivatives, the trapezoid and Simpson rules, orders of accuracy and the error floor from round-off, with runnable Python examples.

Physics is full of derivatives (velocity, force from energy, field from potential) and integrals (work, charge, probability). A computer sees neither limits nor continuous curves; it sees function values on a grid. This lesson turns the two operations of calculus into simple formulas on grid points and shows how to measure their accuracy.

Left: trapezoids under a sine arch. Right: log-log error of forward and central differences versus step size
Trapezoids approximate areas; differences approximate slopes. For derivatives, the error falls with h until round-off takes over.

What you will learn

  • Forward, backward and central difference formulas, and their orders of accuracy.
  • Where the error comes from, using a Taylor series.
  • The trapezoid rule and Simpson's rule.
  • Why the derivative error has a floor, and how to find the best step.

Key terms

Finite difference: an approximation of a derivative using function values at nearby points.

Taylor series: f(x+h) = f(x) + h fโ€ฒ(x) + hยฒ fโ€ณ(x)/2 + hยณ fโ€ด(x)/6 + โ€ฆ

Order of accuracy p: the error scales as h^p; halving h divides it by 2^p.

Quadrature: numerical integration.

Derivatives from Taylor series

Start with the Taylor series f(x+h) = f(x) + h fโ€ฒ(x) + hยฒ fโ€ณ(x)/2 + โ€ฆ and solve for fโ€ฒ(x):

fโ€ฒ(x) = (f(x+h) โˆ’ f(x))/h โˆ’ h fโ€ณ(x)/2 + โ€ฆ

The forward difference (f(x+h) โˆ’ f(x))/h therefore has a leading error โˆ’h fโ€ณ(x)/2, proportional to h: it is first order.

Write the series also for f(xโˆ’h) and subtract the two. The even-power terms cancel:

f(x+h) โˆ’ f(xโˆ’h) = 2h fโ€ฒ(x) + hยณ fโ€ด(x)/3 + โ€ฆ

The central difference (f(x+h) โˆ’ f(xโˆ’h)) / (2h) has leading error hยฒ fโ€ด(x)/6, so it is second order. Halving h cuts the error by 4 rather than 2. Its cost is two function values, the same as forward difference with a fresh start point.

Formula Expression Order
Forward (f(x+h) โˆ’ f(x)) / h 1
Backward (f(x) โˆ’ f(xโˆ’h)) / h 1
Central (f(x+h) โˆ’ f(xโˆ’h)) / (2h) 2
Second derivative (f(x+h) โˆ’ 2f(x) + f(xโˆ’h)) / hยฒ 2
Python Measuring the order of accuracy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each halving of h divides the forward error by about 2 and the central error by about 4. A ratio near 2^p is the practical way to measure the order of any method.

The error floor

If truncation error shrinks as h falls, why not take h = 1e-15? Because the numerator f(x+h) โˆ’ f(x) suffers cancellation. Each function value has a rounding error near ฮต |f|, so the difference has an absolute error near ฮต and the quotient has an error near ฮต/h. The total error is roughly

error(h) โ‰ˆ C h^p + ฮต/h.

The first term falls with h, the second rises. For forward difference the minimum is near h โ‰ˆ โˆšฮต โ‰ˆ 1e-8; for central difference near h โ‰ˆ ฮต^(1/3), a few times 1e-6 (the run below finds about 1e-6 for sin). Below those values accuracy gets worse.

Python Finding the best step size

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Integrals: trapezoids and Simpson

To approximate โˆซ f(x) dx from a to b, split the interval into n strips of width h = (b โˆ’ a)/n. The trapezoid rule replaces each curve piece by a straight line:

T = h [ fโ‚€/2 + fโ‚ + fโ‚‚ + โ€ฆ + f_{nโˆ’1} + f_n/2 ]

Its error scales like hยฒ (second order). Simpson's rule fits a parabola through three points. With n even:

S = (h/3) [ fโ‚€ + 4fโ‚ + 2fโ‚‚ + 4fโ‚ƒ + โ€ฆ + 4f_{nโˆ’1} + f_n ]

Simpson's error scales like hโด (fourth order), so it is far more accurate for smooth functions at the same cost. For an integral that must be right to many digits, scipy.integrate.quad chooses points adaptively.

Python Trapezoid versus Simpson

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Doubling n divides the trapezoid error by 4 and Simpson's by 16. Use these ratios to confirm that your own code achieves the expected order.

Derivatives and integrals of data

When your function is a table of data, use np.gradient(y, x), which applies central differences in the interior and one-sided differences at the ends. For noisy data, differentiating amplifies noise (it divides small differences by h), so smooth first or fit a model. Integration, by contrast, averages noise down, which is why integrating is the safer direction.

Common misconceptions

"Smaller h always gives a better derivative." Round-off error grows like ฮต/h; there is an optimal step.

"Higher order is always better." Higher-order formulas need smooth functions. For a function with a kink, Simpson's rule is no better than the trapezoid rule near the kink.

"The trapezoid rule is exact for straight lines only." True, and that is exactly what makes it second order: the error comes from the curvature fโ€ณ.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. The central difference (f(x+h) - f(x-h)) / (2h) has order

    The even Taylor terms cancel, leaving an error proportional to h squared.
  2. 2. You halve h and the error of a method drops by a factor of about 16. The order is

    2^p = 16 gives p = 4, the order of Simpson's rule.
  3. 3. Why does the derivative error rise again for very small h?

    Cancellation in the numerator is divided by h, so it grows as h shrinks.
  4. 4. Which rule uses parabolas through triples of points?

    Simpson's rule fits a quadratic to every pair of strips.
  5. 5. Roughly where is the best step for a forward difference in double precision?

    Balancing h against epsilon/h gives h near the square root of epsilon, about 1e-8.

Summary

  • Finite differences follow from the Taylor series; central difference is second order, forward only first.
  • Measure the order by halving h and looking at the error ratio.
  • Derivative error has a floor near h โ‰ˆ ฮต^(1/(p+1)) because of round-off.
  • Trapezoid is second order, Simpson fourth; quad is adaptive.
  • Differentiating noisy data amplifies noise; integrating averages it.

Next: Roots and Linear Systems