Increasing scheduled analytic frequency

[pemontto]

We've seen a number of template analytics with search frequency set to 1d. We have some use cases where we'd like to be notified much sooner than the incident + ~1d. What we're struggling with is understanding the ingestion latency for certain log sources. Particularly OfficeActivity where we'd like to know about malicious behaviour within 1-2hrs, but we're seeing ingest latency fluctuations up to 90 minutes based on the query we're looking at which deviates from the data ingestion time documentation:

// Calculate the latency of each log type
union withsource = tt *
| project TimeGenerated, Type
| extend latency = ingestion_time() - TimeGenerated
| where latency > 0s
| summarize
max = max(latency)/1m,
min = min(latency)/1m,
avg = avg(latency)/1m,
p50 = totimespan(percentiles(latency, 50)[0])/1m,
p95 = totimespan(percentiles(latency, 95)[0])/1m by Type
| order by Type asc

Are there any absolute time guarantees for Azure cloud service logs?

[#2 · Accepted solution]

@pemontto 

The link you provided discussed the push from O365 to Sentinel:

"Office 365 solution polls activity logs using the Office 365 Management Activity API, which currently does not provide any near-real time latency guarantees.". 

I'm sure you see from your own query that many tables have acceptable latency.  Its an area always under review and being optimised.

[#3]

@CliveWatson thanks! And thanks for the workbook that wraps everything up nicely, including latency!