Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 37 ยท Errors and Floating Point

Floating-Point Numbers

How computers store real numbers: IEEE 754 doubles, sign, exponent and mantissa, machine epsilon, spacing between floats, overflow, underflow, and inexact decimals.

Computers cannot store most real numbers exactly. There are infinitely many reals but only a finite number of bit patterns, so every number is rounded to the nearest one the machine can represent. This is why 0.1 + 0.2 prints 0.30000000000000004, and it is the source of the rounding errors that every numerical method has to live with.

Python's float (and NumPy's float64) follows the IEEE 754 standard for 64-bit "double precision" numbers. In this lesson you will see exactly how those numbers are built, how big the gaps between them are, and what that means for your algorithms.

How a double is built

A 64-bit double is split into three fields:

Field Bits Role
Sign 1 0 for positive, 1 for negative
Exponent 11 scale (a power of 2, stored with an offset of 1023)
Mantissa (fraction) 52 the significant bits after an implied leading 1

For ordinary numbers the value is

(-1)^sign ร— 1.mantissa ร— 2^(exponent - 1023)

That leading 1. is not stored (it is "implied"), so the 52 stored bits give 53 bits of precision, about 15 to 17 decimal digits.

Python Look at the bits of a double

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

1.0 has an all-zero fraction because it is exactly 1.0 ร— 2^0. But 0.1 has a repeating bit pattern, like 1/3 in decimal. A finite number of bits cannot hold it, so the stored value is a very close neighbour.

Why 0.1 is not exact

0.1 in binary is 0.000110011001100... repeating forever. The stored double is the nearest 53-bit number, which is slightly larger than 0.1. Python can show the exact stored value.

Python Exact value of 0.1 and friends

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Numbers like 0.5, 0.25 and 0.75 are exact because they are sums of powers of 2. Most decimal fractions are not. The lesson for your code is the one from earlier: never compare floats with ==; use a tolerance.

Machine epsilon and spacing

Machine epsilon is the gap between 1.0 and the next larger double. For doubles it is 2^-52 โ‰ˆ 2.22e-16. A related idea is the unit roundoff u = ฮต/2 โ‰ˆ 1.11e-16: the largest relative error made when rounding a real number to the nearest double. So every stored number has a relative error of at most about 1.1e-16.

The gap between neighbours is not constant. It scales with the size of the number: around 1 it is 2.2e-16, around 1e6 it is 1.2e-10, around 1e16 it is 2. That means large numbers have worse absolute precision but the same relative precision.

Python Machine epsilon and spacing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last two lines show a surprising consequence: above 2^53 โ‰ˆ 9.0e15, doubles cannot even represent every integer, so adding 1 may do nothing.

Range: overflow, underflow and special values

The exponent limits the size of a double. The largest is about 1.8e308, the smallest normal positive number is about 2.2e-308.

Event Result
Overflow (too large) inf
Underflow (too small) gradually loses precision (subnormal numbers), then 0.0
0.0 / 0.0, inf - inf nan ("not a number")
1.0 / 0.0 inf in NumPy (a ZeroDivisionError in plain Python)

nan is infectious: any arithmetic with it gives nan, and nan == nan is False. Test for it with np.isnan. When a numerical program suddenly prints nan, look for a zero division, a square root of a negative number or an overflow earlier in the computation.

Python Overflow, underflow and nan

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other precisions

NumPy supports other types, which trade range and precision for memory and speed:

Type Bits Decimal digits Epsilon
float16 16 about 3 9.8e-4
float32 32 about 7 1.2e-7
float64 64 about 16 2.2e-16

Use float64 by default for numerical analysis. float32 is common in graphics and machine learning where speed and memory matter more than the last digits. When you test whether an algorithm is sensitive to rounding, running it in float32 and float64 and comparing is a cheap and useful experiment.

flowchart LR
    A["Real number x"] --> B["Round to nearest double"]
    B --> C["fl(x) = x(1 + ฮด)"]
    C --> D["|ฮด| โ‰ค 1.1e-16"]

The standard model of floating point: every stored number is the true one times (1 + ฮด), with a tiny ฮด.

This model, fl(x op y) = (x op y)(1 + ฮด), is the key to analysing algorithms. Each basic operation (+ - * /) is correct up to one tiny relative rounding error. The next lesson shows how those tiny errors can nevertheless produce a large damage.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does 0.1 + 0.2 not equal 0.3 exactly in Python?

    These decimals have infinite binary expansions, so the stored values are nearby doubles, and their sum differs slightly from the stored 0.3.
  2. 2. What is machine epsilon for float64?

    Machine epsilon is 2^-52, roughly 2.2e-16, the spacing of doubles just above 1.
  3. 3. What does 1e16 + 1 == 1e16 evaluate to, and why?

    The gap between doubles grows with magnitude; near 1e16 it is larger than 1.
  4. 4. What does nan == nan return?

    nan is not equal to anything, including itself. Use math.isnan or np.isnan.
  5. 5. How many bits of precision does a double effectively have?

    52 stored fraction bits plus the implied leading 1 give 53 bits, about 16 decimal digits.

Summary

  • A double has 1 sign bit, 11 exponent bits and 52 fraction bits: about 16 decimal digits.
  • Most decimals (0.1) are not exact; only sums of powers of two are.
  • Machine epsilon is 2.2e-16; each rounding adds a relative error of at most 1.1e-16.
  • Spacing grows with the size of the number; above 2^53 not every integer is representable.
  • Overflow gives inf, invalid operations give nan; never compare floats with ==.

Next: Rounding Error and Cancellation