Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 47 ยท Numerical Foundations

Numerical Foundations: Overview

Preview chapter 2 of the computational physics course: floating point numbers, derivatives and integrals on a grid, root finding, and the error budget that ties them together.

Before you simulate planets or atoms, you need to know what a computer actually does when it adds, subtracts and divides. It stores numbers with a finite number of digits, so almost every result is slightly off. It replaces smooth curves with samples on a grid, so every derivative and integral is an approximation. Good computational physicists are not people who avoid these errors; they are people who know how large the errors are and which one dominates.

Three cards: floating point, calculus on a grid, roots and systems, with the error budget banner
Chapter 2 in one picture: three tools, one budget: answer = truth + truncation error + round-off error.

What you will learn

  • Why the computer's arithmetic is finite, and what that means for results.
  • The two kinds of error: truncation and round-off.
  • What the next four lessons contain.

Key terms

Truncation error: error from replacing an exact operation (a limit, an infinite sum) by a finite one.

Round-off error: error from storing numbers with finite precision.

Order of accuracy: the power p in error โˆ h^p; higher is better.

Machine epsilon: the smallest relative gap between floating point numbers near 1, about 2.2e-16 for doubles.

Two sources of error

Truncation We cut a limit short
Round-off We keep finite digits
1 / 4

A first surprise

Run this and look closely at the output. Nothing is broken: it is exactly how binary floating point works.

Python Why 0.1 + 0.2 is not 0.3

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The decimal number 0.1 has no finite binary expansion, just as 1/3 has no finite decimal expansion. The stored value is the nearest double, off by about 5.5e-18. Chapter 2 shows how such tiny errors can grow, and how to compare numbers safely with a tolerance instead of ==.

The error budget

Every numerical answer can be written

computed answer = true value + truncation error + round-off error

Truncation error falls when you refine the grid (smaller step h). Round-off error usually grows as you refine it, because more operations mean more rounding. The total error is therefore smallest at some intermediate h. This U-shaped curve is the signature of numerical analysis, and you will draw it yourself in the lesson on derivatives.

Python The U-shaped error curve

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What each lesson covers

Common misconceptions

"Floating point errors only matter for huge computations." One subtraction of nearly equal numbers can destroy half the digits. You will see this in the next lesson.

"Higher precision always fixes it." Sometimes a better algorithm fixes it for free, and sometimes extra digits only hide a bad formulation.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which is a truncation error?

    Truncation error comes from cutting an exact limit or infinite sum short.
  2. 2. As the step h gets very small, which error tends to grow?

    Dividing tiny differences amplifies the rounding already present.
  3. 3. What is machine epsilon for double precision, roughly?

    Doubles carry about 16 significant decimal digits.
  4. 4. How should you test whether two floats are 'equal'?

    Rounded numbers rarely match exactly; compare within a tolerance.

Summary

  • Computers store numbers with finite precision and approximate calculus with finite steps.
  • Truncation error shrinks with the step; round-off error tends to grow as the step shrinks.
  • The best step size balances the two, giving a U-shaped error curve.
  • This chapter covers floating point, derivatives and integrals, roots and linear systems.

Next: Floating Point Numbers and Errors