Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 37 ยท Errors and Floating Point

Truncation Error and Taylor Series

Understand truncation error from cutting off a series, use Taylor's theorem to bound it, measure the order of accuracy O(h^p), and balance it against rounding error.

Rounding error comes from the computer; truncation error comes from the method. Whenever a method replaces an infinite or exact process (an infinite series, a limit as h โ†’ 0, a smooth curve) by a finite one, it throws something away, and that is the truncation error. It would exist even on a machine with perfect arithmetic.

The main tool for understanding truncation error is the Taylor series. With it you can predict not just how large an error is, but how fast it shrinks when you refine the method. That second piece of information, the order of accuracy, is how we rank numerical methods.

Taylor's theorem

If f is smooth near x, then for a small step h

f(x + h) = f(x) + h f'(x) + (hยฒ/2) f''(x) + (hยณ/6) f'''(x) + ...

Cut the series after the term of degree n and you get the Taylor polynomial T_n(h). The part you discarded is the remainder, bounded by

|f(x + h) - T_n(h)| โ‰ค M |h|^(n+1) / (n+1)!

where M bounds the (n+1)th derivative between x and x + h. This formula is a guaranteed error bound.

Python Truncate the series for e^x

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Every extra term reduces the error dramatically, because the next term is x^(n+1)/(n+1)! and factorials grow fast. By n = 12 we are at machine precision, and more terms would not help.

Order of accuracy: O(h^p)

For methods with a step size h (a grid spacing, a time step), the truncation error usually behaves like C h^p. We write it O(h^p) and say the method is of order p.

Order If you halve h the error is multiplied by
1 about 1/2
2 about 1/4
4 about 1/16

A forward-difference derivative (f(x + h) - f(x)) / h has truncation error (h/2) f''(x) + ..., so it is first order. Subtracting two Taylor expansions shows that the central difference (f(x + h) - f(x - h)) / (2h) has error (hยฒ/6) f'''(x) + ..., second order.

Python Order of accuracy from halving h

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The "ratio" columns are what you read: about 2 for the forward difference (first order), about 4 for the central difference (second order). Computing the ratio of errors for successive halvings is the standard experiment for checking that you implemented a method correctly. If you expected order 2 and see ratio 2, you have a bug.

You can also estimate the order directly: p โ‰ˆ log2(e(h) / e(h/2)).

The balance with rounding error

Truncation error shrinks as h gets smaller; rounding error does the opposite. In the central difference the numerator f(x + h) - f(x - h) subtracts nearly equal numbers, and dividing by tiny h magnifies the rounding noise by 1/h. The total error therefore has a V-shape: it falls, bottoms out, then rises again.

Python The V-shaped total error

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The error decreases until h is about 1e-5 to 1e-6, then climbs back up. The best step balances the two effects: for a second-order central difference it is about eps^(1/3) โ‰ˆ 6e-6; for a first-order forward difference it is about eps^(1/2) โ‰ˆ 1.5e-8. Chasing h โ†’ 0 blindly makes the answer worse. You will use this balancing argument again in Chapter 6.

flowchart LR
    A["Large h"] --> B["Truncation error dominates<br/>error falls as h falls"]
    B --> C["Optimal h"]
    C --> D["Rounding error dominates<br/>error grows as h falls"]

Truncation error falls with h and rounding error rises; the best h sits between them.

Big-O notation, carefully

g(h) = O(h^p) means |g(h)| โ‰ค C h^p for small h, with some constant C that does not depend on h. It describes behaviour as h โ†’ 0, not an exact value. Two methods of the same order can differ by a large C, and for a fixed practical h a lower-order method with a tiny C can beat a higher-order one. Order tells you how the error scales, not which method is more accurate at one particular step size.

Truncation error also appears in iterations: stopping Newton's method after five steps leaves a truncation (or discretization) error, and so does replacing an integral by a sum of rectangles.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is truncation error?

    Truncation error belongs to the method (cutting a series or a limit); rounding error belongs to the arithmetic.
  2. 2. A method has error O(h^2). If you halve h, the error is roughly

    (h/2)^2 = h^2/4, so the error falls by a factor of about 4.
  3. 3. Why does the error of a finite-difference derivative stop decreasing when h becomes very small?

    Cancellation in f(x+h) - f(x-h) followed by division by a tiny h amplifies rounding error.
  4. 4. Which is the order of the central difference (f(x+h) - f(x-h)) / (2h)?

    The h-linear terms cancel in the Taylor expansions, leaving an h^2 f'''/6 leading error.
  5. 5. Which formula estimates the order p from errors e(h) and e(h/2)?

    If e(h) = C h^p, then e(h)/e(h/2) = 2^p, so p is the base-2 logarithm of the ratio.

Summary

  • Truncation error comes from the method; Taylor's theorem bounds it.
  • A method of order p has error O(h^p): halving h divides the error by 2^p.
  • Verify an implementation by checking error ratios when h is halved.
  • Total error = truncation + rounding, so the best h is not the smallest h.
  • Order says how the error scales, not which method is more accurate at one specific h.

Next: Conditioning and Stability