Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 57 ยท Digital Foundations

Binary Fractions and IEEE 754 Floating-Point

Convert fractions between decimal and binary, learn fixed-point versus floating-point, and decode IEEE 754 single and double precision (sign, exponent, mantissa) in Python.

So far the course has worked with whole numbers: unsigned integers and signed integers in two's complement. Real programs also need 0.5, 3.14159 and 6.02e23. This lesson shows how a computer stores numbers with a fractional part, why 0.1 is never stored exactly, and how the IEEE 754 floating-point format packs a sign, an exponent and a mantissa into 32 or 64 bits.

Converting fractions

Positional notation continues past the binary point with negative powers of two: the first place after the point is worth 1/2, the next 1/4, then 1/8, and so on.

Binary to decimal. 0.101 in binary is 1/2 + 0/4 + 1/8 = 0.625.

Decimal to binary. Convert the integer part by repeated division by 2 (as before) and the fractional part by repeated multiplication by 2. Each multiplication produces the next bit as the integer part of the result, and you carry on with the remaining fraction:

Text
0.625 x 2 = 1.25  -> bit 1, keep 0.25
0.25  x 2 = 0.5   -> bit 0, keep 0.5
0.5   x 2 = 1.0   -> bit 1, keep 0.0  (stop)

Reading the bits top to bottom gives 0.101. For 13.625 you convert 13 to 1101 and the fraction to .101, so 13.625 = 1101.101 in binary.

Some fractions never stop. 0.1 becomes 0.0001100110011... with the block 0011 repeating forever, exactly as 1/3 repeats in decimal. A finite number of bits can only approximate it.

Python Fractions in binary, with repeating patterns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A decimal fraction has a finite binary form only if its denominator, in lowest terms, is a power of two. 0.625 = 5/8 stops; 0.1 = 1/10 does not, because 10 has the factor 5.

Fixed-point numbers

The simplest way to store fractions is fixed point: pick a fixed number of bits for the fraction and treat the integer as if it were scaled. A 16-bit value with 8 fractional bits (written Q8.8) stores x as the integer round(x * 256). Addition and subtraction are ordinary integer operations; multiplication needs a shift to remove the extra scale.

Fixed point is cheap, fast and exact within its resolution, which makes it popular in microcontrollers and signal-processing hardware without a floating-point unit. Its weakness is range: the same number of bits covers either a wide range with coarse steps, or a narrow range with fine steps, never both.

Floating point: scientific notation in binary

Floating point solves the range problem the way scientific notation does. Instead of storing 0.000000123 you store 1.23 x 10^-7: a significand (also called the mantissa) and an exponent. In binary the form is

Text
value = (-1)^sign  x  1.fraction  x  2^exponent

The IEEE 754 standard fixes the layout of the bits:

Format Sign Exponent Fraction Bias Decimal digits
Single (binary32) 1 bit 8 bits 23 bits 127 about 7
Double (binary64) 1 bit 11 bits 52 bits 1023 about 15-16

Three details make the format work.

  1. Normalization. The mantissa is always shifted to the form 1.xxxx. Since the leading 1 is always there, it is not stored; this hidden bit gives 24 bits of precision in single format for the price of 23.
  2. Biased exponent. The exponent field holds the true exponent plus the bias (127 for single). Storing a biased, unsigned number lets floating-point values be compared as if they were integers (for positive numbers).
  3. Special patterns. An exponent field of all zeros or all ones is reserved:
Exponent field Fraction Meaning
all 0s 0 zero (positive or negative)
all 0s not 0 subnormal number (no hidden bit, fills the gap near zero)
1 to 254 (single) any normal number
all 1s 0 infinity (positive or negative)
all 1s not 0 NaN (not a number, e.g. 0/0)

Example: encoding -6.5 in single precision. 6.5 = 110.1 in binary = 1.101 x 2^2. The sign bit is 1. The exponent is 2 + 127 = 129 = 10000001. The fraction (after dropping the leading 1) is 101 followed by zeros: 10100000000000000000000. The whole word is 1 10000001 10100000000000000000000.

Python Decode IEEE 754 single-precision numbers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the row for 0.1. The decoded value is 0.10000000149011612, not 0.1: the nearest 24-bit mantissa. That error is not a bug; it is the repeating binary pattern being cut off.

Rounding and the surprises it causes

Almost every real number must be rounded to fit. IEEE 754 defaults to round to nearest, ties to even. The maximum relative error from one rounding is half a unit in the last place, about 6e-8 for single and 1.1e-16 for double.

The consequences are everyday bugs:

  • 0.1 + 0.2 is not exactly 0.3.
  • Adding a tiny number to a huge one loses the tiny number completely.
  • Summing many numbers gives slightly different results depending on the order.
  • Comparing floats with == is fragile; compare with a tolerance instead.
  • Money should use integers (cents) or a decimal type, not binary floats.
Python Rounding surprises

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other number formats you may meet

  • Half precision (binary16) with 5 exponent bits and 10 fraction bits is common in graphics and machine learning.
  • Decimal floating point (also in IEEE 754) stores digits in base 10, so 0.1 is exact; it is used in finance. See Complements and BCD Arithmetic.
  • Unsigned integers hold 0 up to 2^n - 1 with no sign and no fraction. They are the right choice for counts, sizes, addresses, pixel values and bit masks, and give one extra bit of range compared with signed numbers.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is 0.375 in binary?

    0.375 = 1/4 + 1/8 = 0.011 in binary. Multiplying by 2 gives bits 0, 1, 1.
  2. 2. In IEEE 754 single precision, how many bits are used for the exponent field and what is the bias?

    Single precision uses 1 sign bit, 8 exponent bits (bias 127) and 23 fraction bits.
  3. 3. Why is the leading 1 of a normalized mantissa not stored?

    After normalization the mantissa has the form 1.xxxx, so the first bit carries no information and is left out, giving one extra bit of precision.
  4. 4. Why does 0.1 + 0.2 not equal 0.3 exactly in binary floating point?

    Neither 0.1 nor 0.2 has a finite binary expansion. Each is rounded when stored, and the sum carries both errors.
  5. 5. An exponent field of all 1s with a non-zero fraction represents:

    All-ones exponent with zero fraction is infinity; with a non-zero fraction it is NaN.

Summary

  • Fractions convert to binary by repeated multiplication by 2; fractions whose denominator is not a power of two repeat forever.
  • Fixed point stores a scaled integer: simple and exact, but with a narrow range.
  • IEEE 754 floating point stores sign, biased exponent and a normalized mantissa with a hidden leading 1: single is 1+8+23 bits (bias 127), double is 1+11+52 (bias 1023).
  • Special exponent patterns encode zero, subnormals, infinity and NaN.
  • Rounding explains 0.1 + 0.2 != 0.3; compare floats with a tolerance and keep money in integers.

Next: Signed Binary Numbers.