Handling false positives in Azure Sentinel

[Ofer_Shezaf]

(Updated March 25th, 2021)

When using Azure Sentinel, you are bound to get some false positives. No detection rule is perfect. In this blog post, we will learn how to handle false positives in scheduled analytics rules.

Understanding false positives

Assuming that a detection rule is built correction, false positives often stem from specific entities such as users or IP addresses which should be excluded.  Common scenarios include:

• During normal activity, specific users may exhibit a pattern that can be viewed as suspected. This would often be service principals.
• An intentional security scanning activity coming from known IP addresses is often detected as malicious. 
• A rule excludes private IP addresses. However, some internal IP addresses are not private and should also be excluded. 

In this article, we learn how to resolve false positives using two methods:

1. Using Automation Rules enables creating an exception without modifying the rules and applying the same exception to multiple rules. Also, this method:
   • Works for detections not based on scheduled alert rules.
   • Allows applying an exception on a limited-time basis, for example, while maintenance work triggers false positives that outside of the maintenance window are true incidents. 
   • It keeps a trail, as the exception prevents the creation of an incident, but the alert is still recorded for audit purposes.
2. By modifying the scheduled alert rule, which allows for more elaborate and detailed exceptions. For example, subnet-based exceptions and advanced boolean expressions. Modifying the query also allows using Watchlists to centralize exception management.

Another important distinction is that Automation Rule based exceptions are often generated by analysts, while Alert Rule based exception is typically limited to SOC engineering. 

Adding an exception using an Automation Rule

The simplest way to add an exception is to use the "add automation rule" feature when viewing an incident that is a false positive:

1. Select the incident you want to create an exception for.
2. Click "Create Automation rule."
3. In the sidebar that opens, optionally modify the new rule name to identify the exception, rather than just the alert rule name.
4. Optionally, add additional alert rules for which the exception applies.
5. The sidebar presents the specific entities in the current incident that may have caused the false positive. Keep the automatic suggestion, or modify it to tune better the exception, for example, by changing a condition on an IP address to one checking for an entire subnet.

Now that you defined when the rule triggers, you can continue to define what it does;

1. The rule is already configured to close the incident when the exception criteria are met.
2. You can add a comment that will be added to the automatically closed accident explaining the exception. For example, specify that the incident was closed as it originated from known administration activity.
3. By default, the rule is set to expire after 24 hours automatically. This might be the desired behavior, and in any case, reducing the chance of error. If you would like a permanent exception, set the expiration to a long time period.

All you need to do now is press "apply," and the exception is active.  

As an alternative, you can create an automation rule from scratch by selecting "automation" on the main left side menu and creating a new automation rule.

Adding an exception by modifying an Analytics Rule

Another option is to modify the analytic rule query, either by including the exceptions directly in the rule, or preferably, when possible, including a reference to a watchlist and managing the exception list in the watchlist.

Modifying the query

Taking the typical rule preamble. To implement an exception, you can add the blue line towards the beginning of the query:

let timeFrame = 1d;
SigninLogs
| where TimeGenerated >= ago(timeFrame)
| where IPAddress !in ('10.0.0.8', '192.168.12.1')
…

The relevant exception is not limited to IP addresses and might be for specific users (using the UserPrincipalName field) or Apps (using the AppDisplayName).

You can also exclude multiple attributes, excluding an alert that will include either the IP address 10.0.0.8 or the user "user@microsoft.com":

| where IPAddress !in ('10.0.0.8')
| where UserPrincipalName != 'user@microsoft.com' 

To implement a more fine-grained exception when applicable, and reduce the chance for false negatives, combine attributes. This will create an exception only if both values appear in the same alert:

  | where IPAddress != '10.0.0.8' and UserPrincipalName != 'user@microsoft.com'

The flexibility of in-query exceptions makes it the most complete solution for false positives, at the cost of rules modification.

Excluding subnets

The third use case described above, excluding IP ranges used by the organization, requires subnet exclusion. The following examples show how to exclude subnets. Note that since the ipv_lookup operator is an enrichment operator and not a filtering operator, the filtering is actually done in the following line by inspecting those events for which a match was not made.

let subnets = datatable(network:string) [ "111.68.128.0/17", "5.8.0.0/19", ...];
let timeFrame = 1d;
SigninLogs
| where TimeGenerated >= ago(timeFrame)
| evaluate ipv4_lookup(subnets, IPAddress, network, return_unmatched = true)
| where isempty(network)
…

Using Watchlists to manage exceptions

You can use a Watchlist to manage the list of exceptions outside of the rule itself. When applicable, this is the preferred solutions and has several advantages:

• This enables an analyst to add exceptions without editing the rule, which better follows SOC best practices.
• The same watchlist can apply to multiple rules, enabling central exception management.

Using a watchlist is rather similar to using a direct exception:

let timeFrame = 1d;
let logonDiff = 10m;
let allowlist = (_GetWatchlist('ipallowlist') | project IPAddress);
SigninLogs
| where TimeGenerated >= ago(timeFrame)
| where IPAddress !in (allowlist)
…

Subnets filtering can also be done using a watchlist by replacing in the subnets example above, the subnets table definition with a watchlist:

let subnets = _GetWatchlist('subnetallowlist');

I hope you found this useful!

[bkilroe]

@Ofer_Shezaf thanks for this blog article. It's hugely helpful.

[Pavan_Gelli1910]

@Ofer_Shezaf , Thanks for sharing this article. This is helpful and i came across these situations for few analytics rules. I handled using the | where not(ipv4_is_match("1.2.0.0/17", IPAddress)). Recently i seen similar feature using the Automation rules ,Incident suppression –automatically resolve incidents that are known false/benign positives without the use of playbooks.

Correct me if im wrong in doing correlation bw these features.

[Ofer_Shezaf]

@Pavan_Gelli1910 : you are right, however the feature you mentioned is still in private preview at the time of  writing, hence I did not include it in the post. I will add it and compare the options once automation rules are public.