Forum Discussion
Excel mathematical error
Hello everyone,
I have a very weird mathematical error yet in a very simple context.
In this file : https://1drv.ms/x/c/502badd8e128f93f/Ed9WVtVrU_hDrv16hjNkRv0B1T1r8YLs1QrIUCwGavvIcw?e=vUHKnG the cell H21 is not exactly equal to 0. Which does not make sense at all if you look at the other lines.
What is even more weird, is that if you replace 10 by 1 in the cell H14, the control is exactly 0 as it's supposed to be.
Can anyone explain what is going on ? ![]()
Thanks for reading !
3 Replies
- Mawks69Copper Contributor
Hello Hecatonchire,
Many thanks for the answer and the articles. I read them both and I still have 2 concerns :
In the file there is no division by 3 or anything similar which would result in endless decimals like 3.3333333333333.
Also, the numbers do not go too "high or low" (Microsoft says Excel cannot go beyond ^308). In the file we are speaking about only 10^-6 or -7 at max.
Since the numbers do not go to extremes and there are solely divisions by 10, based on the information you provided I still struggle to understand where does the discrepancy come from.
Ultimately I guess I will have to use the ROUND formula, however I always like to understand things !
- HecatonchireIron Contributor
You haven't read everything.
The 1st example(5.2-5.1)=0.1 return FALSE
of the 2nd link shows the floating point problem with small numbers (problem linked to the conversion of a certain number to binary)