Thank you for liking and using both our ASIM based domain solutions, Network session and DNS essentials. Today, we are announcing the new web session Essentials solution in Public Preview. This is a domain solution and third Microsoft Sentinel solution to leverage Advanced Security Information Model (ASIM). This solution provides a set of generic OOTB (out-of-the-box) content for different products like web server, web proxies and web security gateways that provide web session data currently the solution supports 11 vendor solutions that gathers web session level logs and services which includes Palo Alto Pan OS, Squid proxy, Vectra AI stream, Zscaler internet access, IIS logs, Tomcat Apache and more. This means the same content from this solution can work with multiple web session monitoring products deployed in your organization hence delivering more value to protect your environment with less. Learn more about domain solutions that leverage ASIM.
Microsoft Sentinel has 305+ solutions in Content hub. These solutions enable customers to not only connect their data sources to ingest data in Microsoft Sentinel, but also provide out-of-the-box (OOTB) analytic rules, hunting queries, workbooks, playbooks, and more to help customers realize their E2E scenarios in Sentinel. Even though this approach enables customers to integrate different products in Microsoft Sentinel, there are certain challenges customers face. For example, there are multiple product solutions for the web session domain category, like Palo Alto Pan OS, Squid proxy, Vectra AI stream, Zscaler internet access, IIS logs, Tomcat Apache and more. These have differing data ingest components by design, but there’s a certain pattern to the analytics, hunting, workbooks, etc. within the same category. To take a specific example, most of the major Web session monitoring products have a common basic set of web session alerts that includes URL’s that contain malicious keywords or commands or presence of uncommon user agent in web request. Currently, this analytic rule template is pretty much duplicated for each web session / traffic monitoring category of product solutions. Customers need to check and then configure multiple analytic rules individually if they are running multiple web monitoring products, which is inefficient. Furthermore, this results in alert fatigue when alerts do fire. With the OOTB content built using ASIM, the same alert rule can work across multiple web session monitoring solutions deployed in one's organization.
Key Capabilities: -
- Data normalization using ASIM schema
- Query time or ingestion time parsing
- At scale data / incident handling
- easier use case deployment and incident handling
- More value with less content to manage
- Consolidated workbook views
- Source agnostic content
Prerequisite: -
Web session essentials solution like other Microsoft Sentinel domain solutions don't include a data connector. It depends on the source specific connectors in respective Microsoft Sentinel product solutions to pull in the logs. Install one or more of the prerequisite product solutions listed below. Configure the respective data connectors to meet the underlying product dependency needs and to enable better usage of this solution content.
- Palo Alto Pan OS
- Squid Proxy
- Vectra AI stream
- IIS logs (via legacy agent)
- Tomcat Apache web server
- Fortinet FortiGate
- Barracuda WAF NGFW
- Zscaler Internet Access
- Cisco Meraki
Note: As the parser coverage for this solution increases, this list will also increase.
Out of box content offered: -
This solution comes with fifteen anomaly and threshold based analytic rules, nine hunting queries, one playbook and one workbook,
Analytics rules:
|
Analytic Rule Name |
Description |
|
Detect URLs containing known malicious keywords or commands (ASIM Web Session) |
The utilization of system commands or functions in the request URL may suggest that an attacker is trying to gain unauthorized access to the environment by exploiting a vulnerable service |
|
Detect unauthorized data transfers using timeseries anomaly (ASIM Web Session) |
This rule utilizes built-in KQL anomaly detection algorithms to identify anomalous data transfers to public networks. It detects significant deviations from a baseline pattern, allowing the detection of sudden increases in data transferred to unknown public networks, which may indicate data exfiltration attempts. Investigating such anomalies is crucial. The score indicates the degree to which the data transfer deviates from the baseline value. A higher score indicates a greater deviation. The query's output provides an aggregated summary view of the traffic observed in the flagged anomaly hour, including unique combinations of source IP addresses, destination IP addresses, and port bytes sent. It may be necessary to run queries for individual source IP addresses from the provided 'SourceIPlist' to identify any suspicious activity that warrants further investigation |
|
The download of potentially risky files from the Discord Content Delivery Network (CDN) (ASIM Web Session) |
This detection mechanism identifies instances where requests are made to Discord CDN addresses for file extensions that are considered risky. It triggers when a callout is made to a Discord server that has only been encountered once in the environment. The uniqueness of Discord servers is determined based on the server ID present in the request URL (DiscordServerId in the query). Discord CDN has been utilized in numerous campaigns to download additional payloads, highlighting the importance of monitoring such activities. The query includes a sample set of popular web script extensions (scriptExtensions), which should be customized to align with the specific requirements of your environment |
|
Detect known risky user agents (ASIM Web Session) |
This rule is designed to flag web requests that contain a user agent header that is recognized as malicious. It relies on a predefined list of known user agents, which is referenced from a specific CSV file |
|
Detect Local File Inclusion (LFI) in web requests (ASIM Web Session) |
LFI vulnerabilities allow an attacker to read (and sometimes execute) files on the victim machine. This can be very dangerous because if the web server is misconfigured and running with high privileges, the attacker may gain access to sensitive information |
|
Detect instances of multiple client errors occurring within a brief period (ASIM Web Session) |
This detection mechanism identifies situations where multiple client errors originate from a single source within a limited time frame. |
|
Detect instances of multiple server errors occurring within a brief period (ASIM Web Session) |
This detection mechanism identifies situations where multiple server errors originate from a single source within a limited time frame. |
|
Identify instances where a single source is observed using multiple user agents (ASIM Web Session) |
This detection mechanism identifies requests originating from a single source within a brief time that exhibit multiple user agents. Such behavior could indicate unusual web browsing activities performed by unconventional processes' |
|
Detect potential presence of a malicious file with a double extension (ASIM Web Session) |
Double extension vulnerability is a significant concern in file uploads, as it can lead to various issues if an attacker successfully uploads a virus-infected file. |
|
Detect potential file enumeration activity (ASIM Web Session) |
This detection method identifies potential cases of file enumeration activity. The query is designed to identify client sources that generate multiple requests resulting in 404 error codes' |
|
Detect presence of private IP addresses in URLs (ASIM Web Session) |
'This rule identifies requests made to atypical URLs, as malware can exploit IP addresses for communication with command-and-control (C2) servers. The detection identifies network requests that contain either plain text or Base64 encoded IP addresses. Alerts are triggered when a private IP address is observed as plain text or base64 encoded in an outbound web request. This method of concealing the IP address was observed in the utilization of the RunningRAT tool by POLONIUM.' |
|
Detect presence of uncommon user agents in web requests (ASIM Web Session) |
This rule assists in detecting rare user agents, which may indicate web browsing activity by an unconventional process different from the usual ones. The rule specifically searches for UserAgent strings that have not been seen in the past 14 days. This query will perform better when run over summarized data |
|
Detect requests for an uncommon resource on the web (ASIM Web Session) |
This detection mechanism examines connections made to a domain where only a single file is requested, which is considered unusual since most contemporary web applications require additional resources. Such activity is often associated with malware beaconing or tracking URLs delivered via emails. The query includes a sample set of popular web script extensions (scriptExtensions), which should be customized to align with the specific requirements of your environment' |
|
Detect web requests to potentially harmful files (ASIM Web Session) |
This rule detects web requests made to URLs containing file types such as .ps1, .bat, .vbs,.scr etc. which have the potential to be harmful if downloaded. This rule uses the [Advanced Security Information Model (ASIM)](https://aka.ms/AboutASIM) and supports any web session source that complies with ASIM.' |
|
Detect threat information in web requests (ASIM Web Session) |
This rule would generate an alert if EvenSeverity is 'High' or 'ThreatRiskLevel' or 'ThreatOriginalConfidence' value is greater than 90. |
Hunting queries:
- Empty User Agent Detected (ASIM Web Session)
- Excessive number of forbidden requests detected (ASIM Web Session)
- Detect IPAddress in the requested URL (ASIM Web Session)
- Detect Kali Linux UserAgent (ASIM Web Session)
- Beaconing traffic based on common user agents visiting limited number of domains (ASIM Web Session)
- Potential beaconing detected - Similar sent bytes (ASIM Web Session)
- Potential beaconing detected (ASIM Web Session)
- Request from bots and crawlers (ASIM Web Session)
- Detect threat information in web requests (ASIM Web Session)
Summarization playbook:
The web session essential domain solution is expected to handle data of very high events per second (EPS), and when we have content that is using such high EPS of data there can be some performance impact that can cause slow loading of workbooks or query results. To overcome this, we have created this summarization playbook that will summarize the source logs and store it into a predefined table all the content of essential domain solutions does not query this table unless one has enabled the summarization playbook.
Please be aware that after ‘Summarize web session Data’ playbook is deployed, one must authorize "Azure Monitor Logs" and "Azure Log Analytics Data Collector" API connections. The below screenshot depicts the API connection, which needs to be authorized post playbook installation.
Note: Additional charges might apply for Azure Logic apps. For more information, see the Azure Logic Apps pricing page. Additional charges might also apply for storage of the summarized data.
Workbook:
This workbook is designed for network teams, security architects, analysts, and consultants to monitor, identify and investigate threats on Web servers, Web Proxies and Web Security Gateways assets. This Workbook gives a summary of analyzed web traffic and helps with threat analysis and investigating suspicious http traffic.
The "SummarizeWebSessionData" Playbook installed along with the solution helps in summarizing the logs and improving the performance of the Workbook and data searches. This Workbook leverages the default as well as custom web session summarized data tables for visualizing the data. Although enabling the summarization playbook is optional, we highly recommend enabling it for better user experience in environments with high EPS (events per second) data ingestion. Please note that summarization would require the playbook to run on a scheduled basis to utilize this workbook's capabilities.
This solution provides one workbook web session essentials workbook which provides real-time insights into activity and potential threats in the network.
Web servers
- Events by error type over time
- Top internal users by request count
- Top external users by request count
- Events by Severity
- Top web hosts with most request count
- Top web hosts with most server errors
- Top web hosts with most client errors
- Urls with most failed requests
- Top users with most client errors
- Top users with most server errors
- Rare User Agent requests resulted in success
- Rare User Agent requests resulted in errors
- Top Web servers with highest download
- Possible malicious double extension file upload
Web proxies and security gateways
- Events by products over time
- Events by result over time
- Errors by type over time
- Sent and Received data in GB over time
- Distinct requested applications over time
- Urls with most failed requests count
Top queries
- Top sites of the top users
- Top Users with most request count
- Top Users with most client errors
- Top client error types
- Top websites by successful requests count
- And more...
View threat events
- Events by threat name
- Events by Confidence over time
- Source or Destination IPs matching with Threat Intelligence indicators
- Requested URL matching with Threat Intelligence Indicators
- Source or Destination IPs matching with Entities in Security Alert table
- Request URLs matching with Entities in Security Alert table
- Source HostNames matching with Entities in Security Alert table
Getting started: -
This solution is available on content hub like any other solution. Search the solution and click on install, make sure any of the below listed prerequisite source specific solution(s) are already installed and the respective data connector(s) configured, before installing this solution.
All the content like analytical rule template, hunting query, playbook, workbook can be managed from content hub manage view and will also be available in respective content galleries. Let us know your feedback using any of the channels listed in the questions or feedback section.