Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

- 462K Members
- 7,052 Online
- 559K Conversations

- Home
- :
- Excel
- :
- General Discussion
- :
- FREQUENCY() bug or floating point error?

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

SOLVED
## FREQUENCY() bug or floating point error?

Conversation Options

- Subscribe to RSS Feed
- Mark Conversation as New
- Mark Conversation as Read
- Pin this Conversation for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

06-16-2019 06:18 AM - edited 06-17-2019 05:16 PM

I think this is an easy-to-reproduce bug with frequency() function in Excel. Please also look at the screenshot at the end.

----

*PPS:*

*As the Dellef_Lewin confirmed in his response below towards the end, this seems to be the issue of floating point numbers. **The problem is that even the long-format display mechanism to display more decimals via format->number->decimal places is not correct. So the only way is that we must go to the underlying XML files and verify the numbers explicitly.*

*In summary, this is not the infamous fundamental precision problems with floating point numbers. This is the problem with the design inconsistency among EXCEL's basic functions: IF(), SORT(), RANK() and in the latest, the FREQUENCY() in terms of the number of decimal digits that are handled accurately. For example, IF() and its family handle 15 decimal digits only, where as FREQUENCY and others handle 17 decimal digits. So when IF says two numbers are equal, SORT() or FREQUENCY() may give opposite & surprising results.*

**PS:** @Detlef Lewin @Sergei Baklan

*I have already anticipated the error with the floating point numbers & their accuracy in this post, and I think that it is not the case here. When I displayed the numbers with 30 decimal digits precision on excel itself, I see that the numbers are equal. Note that eventhough IF() comparison confirms the same when compared, the IF() is known to have issues by itself).*

*----*

**Basically, I entered a series of numbers 1,1.1,...,3 using the following steps:**

- Enter 1.0 in cell A1
- Enter the formula (without quotes) "=A1+0.1" in cell A2.
- Fill cells A3:A30 by fill handle

**Create the bins in another column using the following steps:**

- 1.0 in cell C8, 1.5 in cell C9, 2.0 in cell C10, 2.5 in cell C11, 3.0 in cell C12, 3.5 in cell C13, 4.0 in cell C14
- Select cells D8:D15 and press F2, to enter the formula (without quotes) "=frequency(A1:A30,C8:C14)" and finally press Ctrl+Shift+Enter
- While the result is supposed to be 1,5,5,5,5,5,4, you will notice a discrepancy as you may also verify from the screenshot below.

**Notes on debugging attempts & possible causes:**

First of all, note that a non-extreme bin is formed with elements X such that *low < X <= high.*

- The problem is only with the data values that match the bin boundaries. Which often happens in cases, for example, currency data or class marks and so on.
- Because the data is generated using a mathematical formula, and the frequency bins are entered using direct entry, the values are different in the hidden least significant decimal places. As a result, when the bins are tested for less & equal, the value may fall in the wrong bin, resulting in the wrong counts.
- But if this is the correct reason, I should see the mismatched decimal places when I tried to display in long form with 30 decimal places & scientific notation. But I see absolutely no difference in the number (see in the screenshot)
- Further, I performed a conditional test =IF(A21=C12,"YES,Equal","No, not equal") and excel returns that they are equal.
- Finally, I explicitly typed 3.0 (instead of the existing formula) to overwrite the visibly same entry in A21. And the wrong count disappeared and the counting error continued to next boundary. If I overwrite the entry 3.5 by typing it out explicitly (instead of the existing formula), and go on doing it for every bin boundary, the counts are correctly visible.

**Now can someone kindly explain where the problem is?**

My intention is to generate a neat histogram by myself after this basic use of frequency() function to generate the values on x-axis and y-axis.

The numbers, when calculated using the mathematically identical formula (=A1+0.1 etc.) are different from the same result I type them explicitly!

6 Replies

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

06-16-2019 07:51 AM

That's floating point error, but it's not clear, at least for me, why in FREQUENCY. Equivalent formulas like

=COUNTIFS($A$1:$A$30,"<="&C8,$A$1:$A$30,">"&C7) or =COUNTIFS($A$1:$A$30,"<="&C8:C14,$A$1:$A$30,">"&C7:C13) as CSE =SUMPRODUCT(($A$1:$A$30<=C8)*($A$1:$A$30>C7))

etc., all return correct result on this sample, and only FREQUENCY is affected.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

06-16-2019 01:11 PM

The reason for this could be how the numbers are processed in the different functions.

At least COUNTIFS() is known to cut numbers after 15 decimal places.

Ah, found something in my archive:

Calc Bug Due to 17 Digits of Precision

Thanks to MrExcel for solving the case.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

06-16-2019 06:04 PM - edited 06-17-2019 05:23 PM

(EDITED)

Is this a floating point error really? because when I checked numbers displayed in 30 precision digits, it shows correctly!

Probably looking at the underlying XML files is the only way.

Thanks to @Detlef_Lewin for confirming. I can consider the alternative workarounds that use 15 digits only.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

06-17-2019 09:52 AM

SolutionWhen I save the workbook and look into the XML data I see numbers like this:

1.1000000000000001

1.3000000000000003

1.5000000000000004

2.2999999999999998

3.9000000000000026

Definitely floating point error.

Best Response confirmed by
HarishVangala (New Contributor)

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

06-17-2019 02:30 PM

Detlef, thank you for the links (and Bill with Jan Karel for the investigation and explanation of the issue), I had no idea about this before. Here https://sfmagazine.com/post-entry/august-2017-excel-rank-countif-and-floating-point-errors/ is bit more in conclusion part.

In general, using of different precision for SORT, RANK and FREQUENCY is kind of design bug. As well as considering 1900 as leap year and wrong using of negation with exponentiation http://mathforum.org/library/drmath/view/69058.html

Related Conversations

Stable version of Edge insider browser

HotCakeX
in
Discussions
on
10-12-2019
35.1K
Views

7 Likes

35 Replies

flashing a white screen while open new tab

Deleted
in
Discussions
on
10-05-2019
31K
Views

14 Likes

14 Replies

How to Prevent Teams from Auto-Launch

chenrylee
in
Microsoft Teams
on
06-27-2019
180K
Views

8 Likes

29 Replies

Security Community Webinars

Valon_Kolica
in
Security, Privacy & Compliance
on
10-22-2019
16.3K
Views

12 Likes

13 Replies

Share

Popular

Learning Resources

Programs

Values

- Contact us
- Privacy & cookies
- Terms of use
- Trademarks
- About our ads
- © Microsoft