New Detections, Hunting Queries and Response Automation in Azure Firewall Solution for Sentinel

[Mohit_Kumar]

 

Introduction

Recent breaches surface the need for all organizations to adopt an assume breach mindset to security.  While organizations continue to invest heavily in the products and technology to prevent breaches, having automated threat detection and response capabilities to identify malicious actors and actions in your environment has become the need of the hour.  To enable these capabilities at scale, organizations need to have cutting-edge monitoring and response tools along with the detection logic to identify threats.

The cloud native Azure Firewall provides protection against network-based threats.  Azure Sentinel is the cloud native SIEM and SOAR solution which provides threat detection, hunting, and automated response capabilities for Azure Firewall.  While this is great, customers must go through multiple blades and steps in Azure Sentinel to deploy and configure all the detections, hunting queries, workbooks, and automation, which can be an overhead.

Readers of this post will hopefully be aware of the ever-growing integration between Azure Firewall and Azure Sentinel¹.  At Microsoft, we continue to innovate best security detection and response experiences for you, and we are excited to present the Azure Firewall Solution for Azure Sentinel, as announced in the blog post Optimize security with Azure Firewall solution for Azure Sentinel².  The Azure Firewall Solution provides Azure Firewall specific net new detections and hunting queries.  The solution also contains a new firewall workbook and automation components, which can now be deployed in a single, streamlined method.

¹ New Detections for Azure Firewall in Azure Sentinel

¹ Automated Detection and Response for Azure Firewall with the New Logic App Connector and Playbook

² Azure Sentinel Solutions announced in the RSA 2021 conference RSA Conference 2021: New innovations for Azure Sentinel and in the blog post Introducing Azure Sentinel Solutions!

 

Scenario

In case of an attack from an external adversary or malicious activity in a trusted network, the traffic representing the anomaly must inevitably flow through the network where it will be processed and logged by network devices such as Azure Firewall.  While real time threat detection and prevention features such as IDPS etc. can enable you to take actions for the traffic patterns in question ahead of time, there will be scenarios which require a fine gained evaluation before making decisions to block traffic.  This is where Azure Firewall detections and hunting queries in Azure Sentinel provide you with a method to detect threats and respond to them automatically.

 

What’s New

The Azure Firewall Solution provides new threat detections, hunting queries, a new firewall workbook and response automation as packaged content.  This enables you to find the appropriate solution easily and then deploy all the components in the solution in a single step from the Solutions blade in Azure Sentinel.

Below are the details of the components included in the Firewall Solution:

1. New Analytic Rule based detections:
   
   

   Detection rule	What does it do?	What does it indicate?
   Port scan	Identifies a source IP scanning open ports on or through the Azure Firewall.	Malicious scanning of ports by an attacker, trying to reveal open ports in the organization that can be compromised for initial access.
   Port sweep	Identifies a source IP scanning an open port on different IPs through the Azure Firewall.	Malicious scanning of a port by an attacker trying to reveal IPs with specific vulnerable ports open in the organization.
   Abnormal deny rate for source IP	Identifies an abnormal deny rate for a specific source IP to a destination IP based on machine learning done during a configured period.	Potential exfiltration, initial access, or C2, where an attacker tries to exploit the same vulnerability on machines in the organization but is being blocked by the Azure Firewall rules.
   Abnormal Port to protocol	Identifies communication for a well-known protocol over a non-standard port based on machine learning done during an activity period.	Malicious communication (C2) or exfiltration by attackers trying to communicate over known ports (SSH, HTTP) but don’t use the known protocol headers that match the port number.
   Multiple sources affected by the same TI destination	Identifies multiple machines that are trying to reach out to the same destination blocked by threat intelligence (TI) in the Azure Firewall.	An attack on the organization by the same attack group trying to exfiltrate data from the organization.

   
   
   
2. New Hunting Queries:
   
   

   Hunting query	What does it do?	What is it based on? What does it indicate?
   First time a source IP connects to destination port	Helps to identify a common indication of an attack (IOA) when a new host or IP tries to communicate with a destination using a specific port.	Based on learning the regular traffic during a specified period.
   First time source IP connects to a destination	Helps to identify an IOA when malicious communication is done for the first time from machines that never accessed the destination before.	Based on learning the regular traffic during a specified period.
   Source IP abnormally connects to multiple destinations	Identifies a source IP that abnormally connects to multiple destinations.	Indicates initial access attempts by attackers trying to jump between different machines in the organization, exploiting lateral movement path or the same vulnerability on different machines to find vulnerable machines to access.
   Uncommon port for the organization	Identifies abnormal ports used in the organization network.	An attacker can bypass monitored ports and send data through uncommon ports. This allows the attackers to evade detection from routine detection systems.
   Uncommon port connection to destination IP	Identifies abnormal ports used by machines to connect to a destination IP.	An attacker can bypass monitored ports and send data through uncommon ports. This can also indicate an exfiltration attack from machines in the organization by using a port that has never been used on the machine for communication.

   
   
   
3. A single Sentinel Workbook which supports the Azure Firewall Standard and Premium SKUs
   
   
   
4. Custom Logic App Connector and three new Playbooks Templates for Azure Firewall
   
   

   Connector and Playbooks	What does it do?
   Azure Firewall Connector	The connector allows you to take many different actions against Azure Firewall, Firewall Policy, and IP Groups.  A full list of actions supported by the connector is available here
   AzureFirewall-BlockIP-addToIPGroup	This playbook allows you to block IP addresses in Azure Firewall by adding them to IP Groups based on analyst decision.  It allows you to make changes on IP Groups, which are attached to firewall rules, instead of making changes directly to the Azure Firewall.  The target IP Group could be associated with policy/rules used in one or more firewalls
   AzureFirewall-AddIPtoTIAllowList	This playbook allows the SOC to automatically respond to Azure Sentinel incidents which includes a destination IP address, by adding the specific IP to the Threat Intelligence (TI) Allow list in Azure Firewall
   AzureFirewall-BlockIP-addNewRule	This playbook allows you to block an IP address by adding a new network rule with the specific IP to an existing Deny Network Rule Collection in Azure Firewall

Notes:

1. To learn more about the Azure Firewall Logic App Connector and Playbooks, please refer to this blog post: Automated Detection and Response for Azure Firewall with the New Logic App Connector and Playbooks
2. The detections, hunting queries and the firewall workbook included in the solution are based on KQL and you can modify them to meet your specific requirements.  The Firewall Playbooks can also be customized by adding or removing workflows, based on your needs

 

How to Deploy

You must have Azure Firewall Standard or Premium with Firewall Policy or Classic Rules, and Azure Sentinel deployed in your environment to use the solution.  In order to use the response automation capabilities provided by the Azure Firewall Logic App Connector and Playbooks included in the solution, prior to deploying the solution, you must complete the pre-requisites provided in the detailed step by step guide is available here Automated Detection and Response for Azure Firewall with the New Logic App Connector and Playbooks.

Note: You may skip configuration of the Azure Firewall Connector and Playbooks pre-requisites, if you are not planning to use the response automation features at the time of deploying the Firewall Solution

The Azure Firewall solution can be deployed quickly from the Solutions (Preview) gallery in Azure Sentinel.  There are no other prerequisites to deploy and start using the Analytic Rule based detections, Hunting Queries, and the Firewall Workbook included in the solution package.  Please see the screen capture below for a step-by-step process to deploy the firewall solution.

Deploying Azure Firewall Solution for Azure Sentinel

 

How to Configure Solution Components in Azure

After you have successfully deployed the Azure Firewall solution, please use the instructions below to enable and configure the different components of the solution. 

 

Firewall Workbook

Use the following instructions to launch and configure the Azure Firewall Workbook deployed by the solution.

1. Browse over to the Azure Sentinel blade
2. In the left pane, click on the Workbooks node
3. In the Workbooks blade, click on My workbooks tab
4. Click to select the “Azure Firewall” workbook in the My workbooks blade
5. In the right pane (Customer defined workbook), click View saved workbook button

You can now select the appropriate timeframe and firewalls to visualize the logs in the different tabs of the Workbook.

Reference: Visualize your data using Azure Monitor Workbooks in Azure Sentinel | Microsoft Docs

 

Hunting Queries

Use the following instructions to run the Azure Firewall Hunting Queries deployed by the solution.

1. Browse over to the Azure Sentinel blade
2. In the left pane, click on the Hunting node
3. In the Hunting blade, click the checkbox to select one or multiple queries deployed by the solution
   • If you have many preexisting queries, click the Add filter button and then filter on Provider = Custom Queries
4. Click the Run selected queries button on the top to run
5. You will see the results of the query in Results column of the queries

To see detailed results of a query run, click to select the query and click the View results button in the right pane.  This will open the Log Analytics workspace where you can modify the query to drill deeper into the logs.  The query logic can be modified and saved for future use.

Reference: Hunting capabilities in Azure Sentinel | Microsoft Docs

 

Analytic Rules

Use the following instructions to enable and configure the Analytic Rule based detections deployed by the solution.

1. Browse over to the Azure Sentinel blade
2. In the left pane, click on the Analytics node
3. In the Analytics blade, click the checkbox to select one or multiple detection rules deployed by the solution and click the Enable button to enable the detection rule(s)
   • Detection rules deployed by the solution are disabled by default
4. To update the detection logic or the trigger threshold, click to select a detection rule and then click Edit in the right pane
5. The detection logic can be modified in the Set rule logic tab and saved for future use

Now that the solution has been deployed and all components have been enabled/configured successfully, you can use the Firewall Workbook to visualize the Azure Firewall log data, use Hunting queries to identify uncommon/anomalous patterns and create incidents with the enabled detection rules.  You can also automate response for any Azure Firewall detections using the available Azure Sentinel Playbooks.

Reference: Detect threats with built-in analytics rules in Azure Sentinel | Microsoft Docs

 

Testing Detection Rule with Playbook for Automated Response (includes Demo)

In this section, we will use an example scenario to walk you through the steps involved in configuring and testing one of the detections included in the Azure Firewall Solution and respond to it by making the desired update to the Azure Firewall configuration automatically, with one of the Playbooks also included in the solution.  To provide learning aid, a prerecorded end to end demonstration for the scenario is also available at end of this section.  The instructions preceding the demo video are to assist you in setting up and configuring your environment so you can follow along and perform testing based on the scenario outlined below.  We encourage you to follow the step by step process in this section to gain familiarity with key concepts and configuration requirements. 

In the following Example Scenario, you will use the Port Scan rule provided in the solution to detect scanning activity and respond to it automatically using the AzureFirewall-BlockIP-addToIPGroup Playbook.  In this scenario, upon successful detection of a port scan, an incident will be created in Azure Sentinel.  The Playbook will be triggered by the Azure Sentinel Automation Rule which will allow you to add the IP address of the port scanner (source host) to an IP Group used in a deny network rule on Azure Firewall to block traffic from the port scanner.

 

Example Scenario

To test the Port Scan detection and automated response capability, you will need a test environment with:

1. 2 Virtual Machines in separate Spoke VNETs in Azure
2. A Hub VNET with Azure Firewall Standard or Premium which has
   • An Allow Network rule to allow all traffic between the 2 Spoke VNETs
   • A Deny Network rule collection with a Network rule which uses IP Group as the source
3. Ensure that the 2 VMs in Spoke VNETs communicate with each other through the Azure Firewall
   • This can be accomplished by peering the 2 Spoke VNETs where the VMs live with the Hub VNET with Azure Firewall
   • User Defined Routes (UDRs) on the Spoke Subnets to ensure that all traffic from the VMs is routed through the Azure Firewall
4. Azure Sentinel workspace with Azure Firewall Solution deployed and Azure Firewall Connector and Playbooks configured correctly

Here is a diagram of an example setup.  We will be using this setup as reference for the remainder of this document.

 

Configuration Requirements in Example Scenario

Before you can begin testing, please follow the instructions below to ensure Azure Firewall, Azure Firewall Connector and Playbooks (automation) and Azure Sentinel are ready:

 

Azure Firewall

Please ensure that your Azure Firewall has the following configurations:

1. An existing IP Group which will contain IP addresses to be blocked by Azure Firewall
2. An existing Deny Network Rule Collection which is processed before other Allow Network Rule Collections
3. An existing Network Rule in the Deny Network Rule Collection which uses the IP Group from Step 1 for Source

 

Azure Firewall Connector and Playbooks

Please ensure that the Azure Firewall Custom Logic App Connector and Playbooks Templates are configured correctly as described in the detailed step by step guide available here Automated Detection and Response for Azure Firewall with the New Logic App Connector and Playbooks.

 

Azure Sentinel

Please follow the instructions below to configure the Port Scan detection rule and create an automation rule in Azure Sentinel. 

1. Click to select the Port Scan rule and then click the Edit button
2. Click Next: Set rule logic button in the General tab
3. Edit the port scan detection logic in the Rule query pane
4. By default, this rule looks for port scan attempts made 24 hours ago.  To immediately see detection and automated response for a port scan you will be simulating, modify the rule by commenting out the following line in the query
   
   

   //| where TimeGenerated  between (ago(StartRunTime) .. ago(EndRunTime))​

   
   

5. Click on the Next: Incident settings button
6. In the Incident settings tab, click on the Next: Automated response button
7. In the Automated response tab, create a new Automation Rule and attach it to the Port Scan detection rule
   
   1. Click the Add new button in the Incident automation pane
   2. Add a name for the rule in the Automation rule name field
   3. Ensure that the Trigger is set to When incident is created
   4. In Conditions, click to select If Analytic rule name contains Port Scan or Current rule
   5. Under Actions, select Run playbook from the drop-down menu
   6. Select the AzureFirewall-BlockIP-addToIPGroup Playbook from the second drop-down under Actions
   7. Select appropriate Date/time in Rule expiration
   8. Click on the Apply button to create the Automation Rule
8. Click on the Next: Review button in the Automated Response tab
9. Click on the Save button in the Review and create tab to finish rule configuration

Please see the screen capture below for a step-by-step process to modify the Port Scan detection rule and create an Automation rule in Azure Sentinel.

Modifying the Port Scan Detection Rule and creating an Automation Rule

 

Testing Automated Detection and Response in Example Scenario

In the example test setup depicted above, we have a Hub VNET with an Azure Firewall and 2 Spoke VNETs; Client Spoke which has a Kali Linux VM and a Server Spoke which has a Windows Server 2019 VM.  The 2 Spoke VNETs do not have direct connectivity with each other however, both are peered with the Hub VNET and point to Azure Firewall for internet and VNET to VNET connectivity with a UDR (User Defined Route).  Azure Firewall has a Network Rule to allow all traffic from Client Spoke VNET to the Server Spoke VNET.  We have 2 Network rules in Azure Firewall:

• A lower priority rule allows all traffic (all ports and protocols) between the Client and Server Spokes
• A higher priority rule denies all traffic from IP Group used as the source
  
  Note: A higher priority rule is processed before a lower priority rule in Azure Firewall
  
  

We have deployed the Azure Firewall Solution to the Azure Sentinel Workspace and configured the Azure Firewall Connector + Playbooks in this environment.  As described in the previous section (Configuration Requirements in Example Scenario), we have enabled and configured the Port Scan detection rule along with an Automation Rule to trigger the AzureFirewall-BlockIP-addToIPGroup Playbook.  To start the automated detection and response process, we initiate a port scan from the Kali Linux VM in the Client Spoke VNET to the Windows 2019 VM in the Server Spoke VNET using the following command: nmap -Pn -p 1-65535 -v <IP address of the Windows Server 2019 VM> 

Please review the following section to understand all the steps in the automated detection and response flow.

 

How Automated Detection and Response Worked in Example Scenario

The diagram below depicts the end-to-end process starting from the time a port scan is initiated, the Azure Firewall Playbook is triggered based on the detection rule and the IP Group used in the Deny Network Rule in Azure Firewall is updated with the IP address of the port scanner (Kali VM).  All the steps are called out in the diagram and explained below.

1. Port scan is initiated from the Kali Linux VM in the Client Spoke to the Windows Server 2019 VM in the Server Spoke
2. The traffic is routed through the Hub VNET where Azure Firewall processes and allows the traffic based on the Network Rule definition
3. Port scan traffic from the Kali Linux VM in the Client Spoke reaches the Windows Server 2019 VM in the Server Spoke
4. Azure Firewall logs traffic details to the Log Analytics workspace in the Network Rule Log
5. Azure Firewall log data is ingested by Azure Sentinel using the Azure Firewall Data Connector
   
   • Port Scan detection rules in Azure Sentinel analyzes the log data for pattern representing port scan activity
   • When traffic pattern in the log is matched for port scan activity, an Azure Sentinel Incident is created
   • The automation rule attached to the Port Scan detection rule triggers the AzureFirewall-BlockIP-addToIPGroup Playbook
6. The AzureFirewall-BlockIP-addToIPGroup Playbook sends an adaptive notification in the Microsoft Teams Channel defined in its configuration
7. The analyst triaging the incident notification decides to act by adding the IP address of the port scanner host (Kali VM) identified in the notification, to the IP Group used in the deny rule on Azure Firewall
   
   • The Playbook updates the Azure Sentinel Incident with details of action taken
8. The Playbook send the action taken by the analyst to the Azure Firewall Connector
9. The Firewall Connector updates the Azure Firewall configuration by adding the IP address of the port scanner to the IP Group used in the Deny Network rule

Please watch the prerecorded demo below, which shows how to simulate a port scan and walks you through the automated detection and response process in our example scenario.

 

Demo

In this video, we go over the demo environment setup, configuration of Azure Firewall and Azure Sentinel in the demo environment and provide end-to-end demonstration for triggering the automated detection and response process described in the previous section.

 

Summary

The Azure Firewall Solution provides net new detections, hunting queries, workbook and response automation which allow you to detect prevalent techniques used by attackers and malware.  The Solution provides a streamlined method to deploy all packaged components at once with minimal overhead and start utilizing them in your environment.  We encourage all customers to utilize these new detection and automation capabilities to help improve your overall security posture.

We will continue to enhance the firewall solution in the future with new detection and automation capabilities to meet your needs.  You can also contribute new connectors, playbooks, detections, workbooks, analytics and more for Azure Firewall in Azure Sentinel.  Get started now by joining the Azure Network Security plus Azure Sentinel Threat Hunters communities on GitHub and following the guidance.

 

Additional Resources

• To learn more about Azure Firewall, visit:  https://aka.ms/AzNetSecNinja
• To learn more about Azure Sentinel, visit:  http://aka.ms/ninjatraining
• To learn more about Automation Rules and Playbooks, visit:
  • Automate incident handling in Azure Sentinel
  • Automate threat response with playbooks in Azure Sentinel
  • Tutorial: Use playbooks with automation rules in Azure Sentinel