Forum Discussion

adnwilson's avatar
adnwilson
Copper Contributor
Mar 05, 2020

Incorrect 15th decimal causing rounding errors

I have a spreadsheet to create Golf Handicap Tables.  The first column ranges from +5.0 to 54.0 in steps of  0.1  for a total of 591 rows.  The column is displayed to 1 decimal.  But if the same valu...
Formulas and Functions
PReagan's avatar
PReagan
Bronze Contributor
Mar 05, 2020

Hello adnwilson,

 

You have discovered an example of floating point precision. Learn more here:

https://www.microsoft.com/en-us/microsoft-365/blog/2008/04/10/understanding-floating-point-precision-aka-why-does-excel-give-me-seemingly-wrong-answers/

Resources