Azure Sentinel paging on large result set via API

%3CLINGO-SUB%20id%3D%22lingo-sub-2748424%22%20slang%3D%22en-US%22%3EAzure%20Sentinel%20paging%20on%20large%20result%20set%20via%20API%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2748424%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20Community%2C%3C%2FP%3E%3CP%3EWe%20pump%20the%20logs%20of%20Window%20security%20events%20of%20some%20computers%20into%20Azure%20Sentinel%20SIEM.%20Now%20we%20retrieve%20those%20logs%20from%20Sentinel%20to%20local%20database%20by%20using%20REST%20API.%20The%20problem%20is%20when%20the%20result%20set%20is%20large%2C%20the%20API%20return%20error%20message%20like%20%22Result%20size%20too%20large%22.%20So%20we%20want%20to%20implement%20pagination%20and%20fetch%20the%20data%20from%20SIEM%20then%20store%20it%20in%20local%20DB.%3C%2FP%3E%3CP%3EHowever%2C%20according%20to%20MS%20docs%2C%20Kql%20doesn't%20support%20%22Skip%22%20operator.%26nbsp%3B%3C%2FP%3E%3CP%3ESo%20are%20there%20any%20ideas%20how%20to%20implement%20this%20pagination%20method%20to%20fetch%20the%20large%20result%20set%20from%20SIEM%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2748424%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ELog%20Analytics%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
New Contributor

Hi Community,

We pump the logs of Window security events of some computers into Azure Sentinel SIEM. Now we retrieve those logs from Sentinel to local database by using REST API. The problem is when the result set is large, the API return error message like "Result size too large". So we want to implement pagination and fetch the data from SIEM then store it in local DB.

However, according to MS docs, Kql doesn't support "Skip" operator. 

So are there any ideas how to implement this pagination method to fetch the large result set from SIEM?

0 Replies