Watchlists in Microsoft Sentinel allow you to correlate data with events in your Microsoft Sentinel environment. Watchlists can be used for searching, detection rules, threat hunting, and in response playbooks. Built-in watchlist templates integrate with UEBA help to increase fidelity of detections based on anomalous activity. This blog highlights the 4 common use cases for watchlists then goes on to describe sample scenarios associated with each.
Foundational aspects
First off, a few fundamentals around watchlists to help you make the most of them:
Searchkey
In the process of creating your watchlist it is required to specify a SearchKey. The search key is the name of a column in your watchlist that you expect to use as a join with other data or as a frequent object of searches as shown in the sample query below. For more information refer to the documentation.
SigninLogs
| lookup kind=leftouter _GetWatchlist('mywatchlist') on $left.Location == $right.SearchKey
Using the SearchKey together with the lookup operator in the join operation is also recommended to enhance query performance.
Fig. 1 depiction of SearchKey field requirement
_GetWatchlist()
The _Getwatchlist() function is used to retrieve items of a watchlist. It simplifies the usage of watchlists within KQL queries or analytics rules. For example, the function can be used to cache the content of a watchlist and store them in a variable that can be used to represent a sub-query as shown in the example below:
let watchlist = (_GetWatchlist('ipwatchlist') | project IPAddress);
SigninLogs| where IPAddress in (watchlist)
_GetWacthlistAlias()
This function is used to return a list of available watchlist aliases in a workspace. With this information you can determine which ones to use in your analytics rules.
Fig. 2: sample output from execution of _GetWatchlistAlias function
Watchlist table
This is a built-in table within log analytics that holds metadata about Watchlists. This table can be used as a basis to run audit queries as it stores information pertaining to details such as creation, modification or deletion of watchlists. Below is a snapshot of a sample record from the table:
Fig. 3: sample record from the watchlist table
Common Use-cases
Below we highlight five common categories of Use-cases of watchlists followed by sample scenarios for each category:
i. Investigate threats
Watchlists enable you to investigate and respond to incidents quickly with the rapid import of IP addresses, file hashes, and other data from CSV files. After you import the data, use watchlist name-value pairs for joins and filters in alert rules, threat hunting, workbooks, notebooks, and general queries.
Scenario 1: Use a custom Microsoft Sentinel Analytics rule to trigger an alert. The rule query looks up the presence of critical servers in a watchlist against vulnerability data logs imported from Microsoft Defender for Endpoint (MDE).
Fig. i.1: Watchlist snapshot containing critical hosts with alias name “HighValueAssets”
In this scenario, we leverage the built-in High Value Asset template watchlist that contains critical hosts. The vulnerability data logs from MDE has been ingested into Log Analytics custom table named "MDE_TVM_PublicExploits_CL". We then use the KQL operator join to join these two tables to find the matched servers based on the device name.
let watchlst=(_GetWatchlist('HighValueAssets')) //extract watchlist contents and store in variable -> watchlst
| project SearchKey; // specify the searchkey
let secalert=(SecurityAlert //extract hosts from alert and append domain name to match Asset FQDN SearhKey format in the Watchlist
| where TimeGenerated > ago(14d)
| where AlertName contains "Installed software with public exploit detected"
| where Entities has "HostName"
| extend entities = todynamic(Entities)
| mv-apply Entity = entities on (
where Entity.Type == "host"
| extend HostName = Entity.HostName)
| extend AppendDom=strcat(HostName, ".contoso.com"));
secalert
| extend HostName=tostring(HostName)
| join MDE_TVM_PublicExploits_CL on $left.HostName == $right.DeviceName_s // confirm presence of hosts in vulnerability data snapshot
| where TimeGenerated > ago(14d)
| where VulnerabilitySeverityLevel_s == "High" and AppendDom in~ (watchlst) //match hosts with high severity vulnerabilities and presence in HVA watchlist based on SearchKey
| distinct DeviceName_s, VulnerabilitySeverityLevel_s, VulnerabilityDescription_s
ii. Import business data
For example, import user lists with privileged system access, or terminated employees. Then, use the watchlist to create allow lists and blocklists to detect or prevent those users from logging in to the network. Use built-in watchlist templates to leverage UEBA insights in entity pages.
Scenario 2: Use a built-in Analytics rule deployed via the Insider Risk Management Solution to detect suspicious activity by an employee who is about to be terminated
- Create a watchlist template by selecting the Watchlist blade then selecting “Templates” from the resulting menu. Select the “Terminated Employees” template, download the resulting Excel file and populate it with details of terminated employees
Fig. 1.1: snapshot of watchlist template
- Deploy the Insider Risk Management solution from the Microsoft Sentinel Content Hub
Fig. 1.2: snapshot of content hub blade highlighting the Insider Risk Management solution
2. From the Analytics blade, edit the “Insider Risk Management Alert Observed” rule
Fig. 2.1: snapshot of Analytics blade highlighting sample Insider Risk Management rule template
3. Insert the Watchlist alias name into the rule and remove the comment characters (“//”) to enable the line in the KQL query
Fig. 3.1: snapshot of rule logic of sample Insider Risk Management rule template
Ensure to enable the rule as it comes disabled by default.
With the rule created and referencing the built-in watchlist template, incidents can now be generated on this basis. Furthermore, integration with UEBA will make it possible for insights associated with the user to be surfaced in the Entity pages per below screenshot:
Fig. 3.2: snapshot of user entity page showing related watchlist insights
iii. Reduce alert fatigue.
Create allow lists to suppress alerts from a group of users, such as users from authorized IP addresses that perform tasks that would normally trigger the alert. Prevent benign events from becoming alerts.
Scenario 3: Automate closure of Microsoft Sentinel incidents triggered by IPs from known sources.
Below are high-level steps of how we can use a logic app to query the contents of an existing watchlist then make a verdict on how to treat a triggered incident as well as add some enrichment to the incident.
This playbook is available here in the Microsoft Sentinel GitHub repository and can be adapted to address similar Use-cases.
Fig. 4.1: snapshot of high level view of watchlist update logic app
In the above image the “Query” section shows how we leverage KQL to look up the watchlist containing known IPs. If any IPs are matched between what is in the alert and the contents of the specified Watchlist, then the automation branches to the step below which will dismiss the incident, thereby saving analysts from expending time to investigating it.
Scenario 4: Leverage a scheduled rule that looks for anomalous user behavior and adds confidence to the alert by matching the user against a watchlist
Assuming the Insider Risk Management solution referred to in scenario 2 above has been deployed, you can leverage one of the anomaly detection rules to increase fidelity of a triggered alert. This is because by definition anomaly-based detections are low fidelity alerts and therefore could result in a high volume of false positives. However, by using watchlists together with these rules; one can increase fidelity of resulting incidents, thereby reducing alert fatigue.
As an example, the “High Risk User Security Alert Correlations” rules triggers alerts through joining the SecurityAlert from Microsoft Products with SecurityIncident in Microsoft Sentinel and Microsoft 365 Defender. This join identifies patterns in user principal names associated with respective security alerts. A machine learning function (Basket) is leveraged with a .001 threshold. Basket finds all frequent patterns of discrete attributes (dimensions) in the data. It returns the frequent patterns that exceeded the frequency threshold. Adding a watchlist to the template will limit alerts to be triggered only when matched against specific identities within the watchlist as depicted below:
Fig. 4.2: snapshot of Insider Risk Management rule template
iv. Enrich event data
Use watchlists to enrich your event data with name-value combinations derived from external data sources.
Scenario 5: Leverage the custom details feature in Analytics rules to surface specific alert details in order to refine subsequent automation actions and add more context to critical alerts
The custom schedule rule below will query the contents of a watchlist in order to determine whether this alert originated from a test environment. If this is confirmed to be the case, then the resulting incident will be dismissed and comments added via a logic app. If the alert is indeed from a production environment, then the incident will be triggered and will contain the server name, giving it more prominence to the analyst.
Fig. 5.1: snapshot of custom analytics rule template
Configuring custom details enrichment
Fig. 5.2: snapshot of custom property page if analytics rule
Resulting alert/incident generated from the rule:
Fig. 5.3: snapshot of alert with servername as custom property
KQL query within logic app to reference watchlist to determine subsequent treatment of the incident, one of the goals being reduction of false positives.
Fig. 5.4: snapshot of KQL query referencing watchlist from within a logic app
Summary
With these scenarios that leverage watchlists working in concert with other features in Microsoft Sentinel, we provide you with a range of options to increase efficiency in your SOC. Try out the scenarios that suit you and provide us with feedback so we can continually improve the product for your benefit.
Related resources:
What is a watchlist - Microsoft Sentinel | Microsoft Docs
Build queries or rules with watchlists - Microsoft Sentinel | Microsoft Docs
Microsoft Sentinel Large Watchlist
join operator - Azure Data Explorer | Microsoft Docs
