Forum Discussion
Ron_Hockman1525
Mar 22, 2023Copper Contributor
Excel Math
I am not sure why this math is off. The formulas are column * 83.50 to get column 2 (time units) Then Column 6 multiplies column 2 * 83.50 + Column 3 (20). .87*83.5+20 = 92.645 not 92.37. What am...
- Mar 22, 2023
Excel follows the IEEE 754 specification on how to store and calculate floating-point numbers.
Excel therefore stores 15 significant digits in a number, and changes digits after the fifteenth place to zeroes.
This could be why you are seeing a difference between your calculator and Excel.
Recommend to use the ROUND function to get the same result.
Hope I could help you with these information / links.
I know I don't know anything (Socrates)
NikolinoDE
Mar 22, 2023Gold Contributor
Excel follows the IEEE 754 specification on how to store and calculate floating-point numbers.
Excel therefore stores 15 significant digits in a number, and changes digits after the fifteenth place to zeroes.
This could be why you are seeing a difference between your calculator and Excel.
Recommend to use the ROUND function to get the same result.
Hope I could help you with these information / links.
I know I don't know anything (Socrates)
Ron_Hockman1525
Mar 22, 2023Copper Contributor
Thank you