Mar 05 2020 07:26 AM
I have a spreadsheet to create Golf Handicap Tables. The first column ranges from +5.0 to 54.0 in steps of 0.1 for a total of 591 rows. The column is displayed to 1 decimal. But if the same values are displayed to 16 decimals, an extraneous digit is appearing in some rows in the 15th decimal place. Calculations done on the values in that column can produce results like 7.49999999999999 instead of 7.5000000 - that causes rounding down to 7.4 instead of 7.5. It's easy enough to fix, but if you don't know it's happening, you can get incorrect results as I did. Attached spreadsheet demonstrates the problem. I can see no pattern in the way the unwanted digits appear.
Mar 05 2020 07:32 AM - edited Mar 05 2020 07:33 AM
This is a question that has been asked many times before. See this article:
https://docs.microsoft.com/en-us/office/troubleshoot/excel/floating-point-arithmetic-inaccurate-resu...
Mar 05 2020 07:32 AM
Mar 05 2020 07:54 AM