Forum Discussion

Ron_Hockman1525's avatar
Ron_Hockman1525
Copper Contributor
Mar 22, 2023

Excel Math

I am not sure why this math is off.  The formulas are column * 83.50 to get column 2 (time units)  Then Column 6 multiplies column 2 * 83.50 + Column 3 (20).  .87*83.5+20 = 92.645 not 92.37.  What am...
  • NikolinoDE's avatar
    Mar 22, 2023

    Ron_Hockman1525 

    Excel follows the IEEE 754 specification on how to store and calculate floating-point numbers. 

    Excel therefore stores 15 significant digits in a number, and changes digits after the fifteenth place to zeroes.

    This could be why you are seeing a difference between your calculator and Excel.

     

    Recommend to use the ROUND function to get the same result.

     

    Hope I could help you with these information / links.

     

    NikolinoDE

    I know I don't know anything (Socrates)

Resources