# Why 0.1 + 0.2 ≠ 0.3: A Deep Dive into IEEE 754 and Floating-Point Arithmetic

> ***You can find the Chinese version at* 從** [**IEEE 754 標準來看為什麼浮點誤差是無法避免的**](https://medium.com/starbugs/see-why-floating-point-error-can-not-be-avoided-from-ieee-754-809720b32175)

### Introduction

When people first learn to code, they often encounter floating-point errors. If you haven't experienced this yet, you're very lucky!

For example, in Python, 0.1 + 0.2 does not equal 0.3, and 8.7 / 10 does not equal 0.87 but 0.869999… It's really strange.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720534179733/fde695a7-25c3-424e-9959-c1a915e863e4.webp align="center")

However, this is not a bug or a flaw in Python. It's the result of how floating-point numbers are calculated, and this happens in other languages like Node.js too.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720534195039/ba85c649-d4d0-4c2d-b508-11e8fd8a86af.webp align="center")

### **How Computers Store Integers**

Before discussing floating-point errors, let's look at how computers represent integers using binary (0s and 1s). For instance, `101` in binary represents 2² + 2⁰ = 5, and `1010` represents 2³ + 2¹ = 10.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720536037662/b1becaf6-6698-410b-9438-0507f0049ec9.jpeg align="center")

A 32-bit unsigned integer can hold 32 bits, so the smallest value is `0000...0000` (0), and the largest is `1111...1111`, which is 2³¹ + 2³⁰ + … + 2¹ + 2⁰ = 4294967295.

Each bit can be either 0 or 1, giving 2³² possible values. <mark>This means any value between 0 and 2³²-1 can be represented without error.</mark>

### **How about Floating-Point Numbers**

While there are many integers between 0 and 2³²-1, their number is limited to 2³². Floating-point numbers are different. <mark>In the range of 1 to 10, there are only ten integers but an infinite number of floating-point numbers like 5.1, 5.11, 5.111, etc.</mark>

Since a 32-bit space has only 2³² possibilities, fitting all floating-point numbers into this space requires a standard representation, which is where [IEEE 754](https://en.wikipedia.org/wiki/IEEE_754) comes in.

## **IEEE 754 Standard**

IEEE 754 defines how to represent single (32-bit), double (64-bit), and special values (infinity, NaN) for floating-point numbers.

### Normalization

For example, to convert `8.5` to IEEE 754 format, you normalize it by breaking it into `8 + 0.5`, which is `2³ + 1/2¹`. In binary, this is `1000.1`, or `1.0001 x 2³`, similar to scientific notation in base 10.

### **Single Precision Floating-Point**

In IEEE 754, a 32-bit floating-point number has three parts: sign, exponent, and fraction, adding up to 32 bits.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720536071339/6c9b7bb7-9a2a-48ee-ba98-651d367e1746.jpeg align="center")

* **Sign**: The leftmost bit indicates the sign (0 for positive, 1 for negative).
    
* **Exponent**: The next 8 bits represent the exponent in a biased format (value + 127).
    
* **Fraction**: The last 23 bits represent the fractional part (after the decimal point).
    

For example, 8.5 in 32-bit format is represented as:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720535915626/2c2796ab-e677-4ea0-a5a8-b1b948cb9f35.jpeg align="center")

### **When is it Inaccurate?**

<mark>The example of 8.5 can be precisely represented as 2³ + 1/2¹ because both 8 and 0.5 are powers of 2, without precision loss.</mark>

However, for 8.9, which cannot be exactly represented as a sum of powers of 2, it must be approximated, resulting in small errors. You can explore these errors using the [IEEE-754 Floating Point Converter](https://www.h-schmidt.net/FloatConverter/IEEE754.html).

### **Double Precision Floating-Point**

To reduce errors, IEEE 754 also defines a 64-bit representation, doubling the size of the fraction part from 23 to 52 bits, thereby increasing precision.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720536101159/b0e913f5-4bef-44b7-aa83-119e63875c2f.jpeg align="center")

For instance, while 8.9 still can't be perfectly represented, the error is much smaller in 64-bit format compared to 32-bit.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720535297759/34e5ade4-643d-4ca0-a151-c1ca67059a81.webp align="center")

In Python, 1.0 and 0.999...999 are considered equal, as are 123 and 122.999...999, because their difference is too small to be represented in the fraction part.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720535304404/0795e68b-8595-47e4-956e-2f8f142ea71e.webp align="center")

## **Solutions**

Since floating-point errors are unavoidable, here are two common ways to handle them:

### **Set a Maximum Allowable Error (epsilon)**

Some languages provide an epsilon value to determine if a result is within an acceptable error range. In Python, epsilon is about 2.2e-16.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720535396586/56c9f0fe-fe02-4bfe-a4e0-90812521ee17.webp align="center")

You can rewrite `0.1 + 0.2 == 0.3` as `abs(0.1 + 0.2 - 0.3) <= epsilon` to avoid errors in calculations.

### Use Decimal Calculations

<mark>To avoid conversion errors between decimal and binary, use decimal calculations directly.</mark> Python's `decimal` module can handle this, performing exact decimal arithmetic.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1720535511559/ae90ce56-97e0-4bd2-8b68-307ac07df59a.webp align="center")

While using decimal arithmetic eliminates floating-point errors, <mark>it is slower because it simulates decimal operations on the CPU, which natively uses binary.</mark>

## Conclusion

Floating-point errors are unavoidable, but understanding IEEE 754 can help you manage them. Knowing the format's details can be interesting and useful, allowing you to explain why these errors occur confidently.

### References

* [IEEE 754](https://en.wikipedia.org/wiki/IEEE_754)
    
* [IEEE-754 Floating Point Converter](https://www.h-schmidt.net/FloatConverter/IEEE754.html)