Forum Discussion
Historical IOC searches
Hello everybody,
I'm interested to understand how others approach this and what is perhaps considered the best practice for performing historical searches for IOC hits against the log data within Sentinel.
For example; if a request is received to interrogate the previous 6 months worth of retained log data against a large list of IOC IP addresses what method is best suited for this?
Currently I am creating KQL queries and running these against the appropriate tables, or all tables if this is required. However these queries time out and end after circa 10 minutes so this is not always practical for large investigations.
Additionally construction of the KQL queries for multiple IOC values is time consuming as you have to manually populate the query string with the relevant IOC and Sentinel KQL operator, using find and replace for example then pasting this back. Is there not a way like other SIEMs where you can create a list of IOCs (IP addresses or domains etc) and then reference that list within the KQL as not to have to manually construct the query on each occasion you perform your retrospective searches?
Thanks in advance for your help and comments.
- Generally if you need a list you have two main choices, creating the IP Addresses or IOC in a Watchlist (or on Azure Storage) or creating a dynamic list as part of the query
https://docs.microsoft.com/en-us/azure/sentinel/watchlists
KQL example of a dynamic list
let IPList = dynamic(["216.24.185.74", "107.175.189.159","194.88.106.146"]);
ThreatIntelligenceIndicator
| where NetworkSourceIP in (IPList)
| summarize count() by NetworkSourceIP
Having a timeout 'suggests' there is some optimisation we can do, do you use the summarize command? Also see https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/best-practices
- CliveWatson
Microsoft
Generally if you need a list you have two main choices, creating the IP Addresses or IOC in a Watchlist (or on Azure Storage) or creating a dynamic list as part of the query
https://docs.microsoft.com/en-us/azure/sentinel/watchlists
KQL example of a dynamic list
let IPList = dynamic(["216.24.185.74", "107.175.189.159","194.88.106.146"]);
ThreatIntelligenceIndicator
| where NetworkSourceIP in (IPList)
| summarize count() by NetworkSourceIP
Having a timeout 'suggests' there is some optimisation we can do, do you use the summarize command? Also see https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/best-practices- Thanks Clive, your help was appreciated. I have managed to utilise Watchlists and referenced this within the query as you said.
- You can also look at integrating your IOC list with the Microsoft Security Graph - https://docs.microsoft.com/en-us/graph/api/resources/tiindicator?view=graph-rest-beta
Then they will show in the ThreatIntelligenceIndicator table.
You could also look at ingesting them to a custom table - https://docs.microsoft.com/en-us/rest/api/loganalytics/
Guess it depends how dynamic that list is, if it is a once off investigation then a watchlist is probably the easiest/most effective, if that list updates more often then I would go one of the other two options.
