Forum Discussion
RobYoung
Aug 25, 2023Iron Contributor
Missing data from the Office Activity logs
I run a query on a daily basis that uses the OfficeActivity table and filters the term Send within the operation field. I started to notice that my results were decreasing so I ran a summary for the past month and noticed a huge decrease in OfiiceActivity capturing the send activity. Any thoughts on what would be the cause of this?
PS it is not sentinel missing data, because when I check the activity in Defender for cloud, the results are the same.
Here is the query I ran:
OfficeActivity
| where TimeGenerated > ago(30d)
| where Operation contains "Send"
| summarize count() by bin(TimeGenerated, 1d)
And here are the results:
TimeGenerated [UTC] | count_ |
8/25/2023 | 417 |
8/24/2023 | 66 |
8/23/2023 | 93 |
8/22/2023 | 77 |
8/21/2023 | 73 |
8/20/2023 | 16 |
8/19/2023 | 17 |
8/18/2023 | 326 |
8/17/2023 | 2978 |
8/16/2023 | 3175 |
8/15/2023 | 4106 |
8/14/2023 | 3632 |
8/13/2023 | 466 |
8/12/2023 | 527 |
8/11/2023 | 2516 |
8/10/2023 | 3187 |
8/9/2023 | 3143 |
8/8/2023 | 3289 |
Now today it is looking like it is starting to climb back but I need to rely on this data so I wouldn't mind knowing why it stopped for almost a week. (no changes that would impact our environment were made btw)
- G_Wilson3468Iron Contributor
RobYoung There could be a few things going on here. If you are not missing data then I would look at sampling data issues. For example, your time range is large. 30 days of processing will involve more data to process and can result in less accurate results. I suggest you reduce the "TimeGenerated" to 15 days and then set your aggregation to 7 days instead of 1. This will also reduce the whole number of data points, doing so take a greater load off the processing and can help with sampling accuracy.
Hope this helps
G.
- RobYoungIron ContributorThis issue magically fixed itself shortly after posting. The sampling I usually due on a 24 hour bases, it was only for the purpose of the post above to show how it fluctuated from day to day. Thanks for the follow up though