Floating Point Rounding Errors

Tags: FAQData

Floating Point Rounding Errors

Generally, “floating point error” refers to when a number cannot
be stored in the IEEE floating point representation.

Integers are stored with the right-most bit being 1, and each
bit to the left being double that (2,4,8,…). It’s easy to see that this can
store any integer up to 2^n, where n is the number of bits.

The mantissa (decimal part) of a floating point number is stored
in a similar way, but moving left to right, and each successive bit being half
of the value of the previous one.

Thus, numbers like 0.5 (1/2) are easy to store, but not every
number <1 can be created by adding a fixed number of fractions of the form
1/2, 1/4, 1/8, …

A really simple example is 0.1, or 1/10. This can be done with
an infinite series, but whenever
a computer stores 0.1, it’s not exactly this number that is stored.

Content Created by David Franzwa
Scroll to Top