Forum Discussion
Sentinel incidents API: createdTimeUtc filter missed incidents that showed up later
On one day we saw four incidents created between about 09:30 and 10:45 UTC that were not returned by our polls. Each poll covering their creation times got HTTP 200 with an empty value list (no errors, no throttling). When we ran the identical query later, with the same time window and api-version, all four were returned.
Details:
- Two of the incidents came from Microsoft Defender XDR (providerName: Microsoft XDR). The other two came from a Sentinel analytics rule.
- Normally, incidents show up in our polls within about 3 minutes (median) and never more than 15 minutes, based on a week of our logs.
- The Microsoft documentation says Defender XDR incidents "typically appear in the Microsoft Sentinel UI and API within five minutes". Ours took far longer.
Questions:
- Is the Incidents List API with a createdTimeUtc filter eventually consistent? What delay should clients expect, and is there an upper bound?
Can an incident appear in the API after its createdTimeUtc has already passed, for example when synced from Defender XDR?
Note: Currently we have poll every 5 minutes, with overlapping ~20-minute windows.
1 Reply
I do not know of a documented upper bound for how late an incident can appear, so I would not rely on one. What I can say is why a createdTimeUtc filter is fragile for this: that timestamp is when the incident was created at its source (for Defender XDR incidents, when it was created there), and the record reaches Sentinel afterwards. So an incident can show up in the API after its createdTimeUtc has already passed, and a poll that only looks at a recent creation window can miss it, even with overlap. Your 20 minute windows did not help here because the delay was longer than the overlap.
Ways to make the poller robust, in the order I would try them:
- Filter on lastModifiedTimeUtc and keep a watermark. An incident that is synced in late also gets a modified time later, so a query for lastModifiedTimeUtc greater than your last watermark (minus a safety margin) catches new incidents and changes to old ones.
- Use the SecurityIncident table in Log Analytics instead, and filter on ingestion_time() greater than your watermark. Ingestion time is the right field for catching late arrivals, and you can dedupe on IncidentNumber and take the latest row per incident.
- Detect gaps. Incident numbers increase in your workspace, so if your poll sees numbers 100, 101 and 103, you know 102 is missing and you can re-query it later instead of assuming it never existed.
- If you want events, not polling, use an automation rule or playbook on incident creation to push to your system (Logic App, or Event Hub), and keep a slow reconciliation poll as the safety net.
For the four missed incidents, I would open a support case with their incident numbers, creation times and the time you first saw each one. The five minute figure in the docs is a typical value, and the exact timings of those four give the Sentinel team something concrete to check.
