Forum Discussion

EExmannhotmailcom's avatar
EExmannhotmailcom
Copper Contributor
Jul 05, 2024

Calculation error

Hello, I have a matrix of numeric values 13 by 7 cells. If I summarize the horizontal figures (13 columns in total) it gives me a total. If I do the same (different purpose) for the vertical (7 ro...
  • EExmannhotmailcom's avatar
    Jul 10, 2024

    EExmannhotmailcom 

    Hello Joe User,

     

    Thanks for your reply's, don't know why I received your request (13 times) for concrete examples.

     

    You clarified one thing for me,

    "The actual limitation is the internal binary precision, not a decimal precision.

    Second, the formatting limit is 15 significant digits, not decimal places.

    For example, 12345.6789012345 has only 10 decimal places, but it has 15 significant digits, just as 0.123456789012345 does."

     

    That is the point where my calculations go wrong, but since I added a check, which notify’ s me if there is a difference in the total horizontal compared to the total vertical. It makes it easier for me to rule out any problems.

    I have added the example you requested. Hope this clarifies it for you.

Resources