Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 37 ยท Errors and Floating Point

Absolute and Relative Error

Learn how to measure the size of an error with absolute error, relative error, percent error and significant digits, and how to pick the right one for your problem.

A numerical answer is only useful if you know how wrong it might be. Saying "the root is 1.414" tells you little; saying "the root is 1.414 with an error below 0.001" tells you everything. This lesson introduces the standard ways to put a number on error: absolute error, relative error, percent error and significant digits.

Suppose the true value is x and your approximation is xฬ‚ (read "x hat"). Every measure below compares those two numbers; they differ in how they treat the size of x.

Absolute and relative error

The absolute error is the plain distance between the two values:

E_abs = |x - xฬ‚|

The relative error divides by the size of the true value:

E_rel = |x - xฬ‚| / |x| (for x โ‰  0)

Multiply the relative error by 100 for percent error.

Why have both? An absolute error of 1 metre is huge when measuring a table and nothing when measuring the distance to the Moon. The relative error captures that: 1 m out of 2 m is 50%, 1 m out of 384,000,000 m is about 0.0000003%.

Python Absolute versus relative error

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the first two rows the absolute error is identical (1), but the relative errors differ by eight orders of magnitude. For comparing methods, relative error is usually the fairer measure. The exception is when the true value is zero or very close to zero: then relative error blows up (or is undefined), and you fall back on absolute error or a combination.

Significant digits

A common way to describe accuracy is significant digits (also called significant figures): digits that carry meaning, counted from the first non-zero digit. 0.00314 has three significant digits; 31400 has three (or five, depending on whether the trailing zeros are measured).

The link with relative error is direct. An approximation has about n correct significant digits when

E_rel < 0.5 * 10^(1 - n) or, as a rule of thumb, n โ‰ˆ -log10(E_rel).

So a relative error of 1e-6 means about 6 significant digits.

Python Count correct significant digits

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the answer is a rule of thumb, not an exact count: 3.14 matches pi in three digits and the formula gives 3.3.

Errors you can bound but not compute

In practice you rarely know the true value (otherwise why approximate?). Then you cannot compute E_abs directly, but you can often get an error bound or an estimate. Two typical situations:

  1. A guarantee from theory. For bisection, after n steps the root lies within an interval of width (b - a) / 2^n, so the error is at most that.
  2. An estimate from two approximations. If you computed xฬ‚_n and a better xฬ‚_(n+1), then |xฬ‚_(n+1) - xฬ‚_n| estimates the error of xฬ‚_n. Iterative methods stop on exactly this quantity.

Be careful with words. An error bound is a guarantee (the error is never larger). An error estimate is a good guess (the error is probably about this size). Do not present one as the other.

Propagation of error

When you do arithmetic on approximate numbers, their errors combine. For small relative errors:

Operation Rule
a + b or a - b absolute errors add (worst case)
a * b or a / b relative errors add (worst case)
a^n relative error is multiplied by n
f(a) abs error โ‰ˆ `

The last row is the general rule and is the seed of the idea of conditioning, which comes later in this chapter.

Python Propagation of error through a product

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The worst-case relative error (about 3.02%) is close to the sum of the two side errors (1% + 2% = 3%). The tiny extra comes from the product of the two small errors, which the rule ignores.

Choosing a tolerance in code

In a program you will compare numbers many times, for instance in tests. A good combined test uses both absolute and relative tolerance:

Python
import math

def close(x, y, rel_tol=1e-9, abs_tol=1e-12):
    return abs(x - y) <= max(rel_tol * max(abs(x), abs(y)), abs_tol)

print(close(1e10 + 1, 1e10))       # True: relative gap is 1e-10
print(close(1e-15, 2e-15))         # True: only the abs_tol keeps this sane
print(math.isclose(1e-15, 2e-15))  # False: default abs_tol is 0

Without abs_tol, two numbers that should both be "essentially zero" never compare equal. Remember this whenever a result can legitimately be zero.

Reporting an error with false precision is a classic mistake. If your error is about 0.003, say "1.414 plus or minus 0.003", not "1.41421356 plus or minus 0.003".

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. The true value is 200 and the approximation is 198. What is the relative error?

    Absolute error is 2, divided by 200 gives 0.01 (1 percent).
  2. 2. When is relative error a poor choice?

    Dividing by a tiny or zero true value makes the ratio huge or undefined, so use an absolute error instead.
  3. 3. A relative error of 1e-6 corresponds to roughly how many correct significant digits?

    The number of correct digits is about -log10 of the relative error.
  4. 4. When multiplying two approximate numbers, how do the worst-case errors combine?

    For products and quotients the relative errors add (to first order).
  5. 5. What is the difference between an error bound and an error estimate?

    A bound is guaranteed never to be exceeded; an estimate is only likely to be about right.

Summary

  • Absolute error |x - xฬ‚| has the units of the quantity; relative error divides by |x| and has no units.
  • Significant digits โ‰ˆ -log10 of the relative error.
  • When the truth is unknown, use theoretical bounds or compare successive approximations.
  • Absolute errors add in sums, relative errors add in products.
  • Compare floats in code with a combined absolute and relative tolerance.

Next: Floating-Point Numbers