Microsoft Sentinel’s AI-driven UEBA ushers in the next era of behavioral analytics
Co-author - Ashwin Patil Security teams today face an overwhelming challenge: every data point is now a potential security signal and SOCs are drowning in complex logs, trying to find the needle in the haystack. Microsoft Sentinel User and Entity Behavior Analytics (UEBA) brings the power of AI to automatically surface anomalous behaviors, helping analysts cut through the noise, save time, and focus on what truly matters. Microsoft Sentinel UEBA has already helped SOCs uncover insider threats, detect compromised accounts, and reveal subtle attack signals that traditional rule-based methods often miss. These capabilities were previously powered by a core set of high-value data sources - such as sign-in activity, audit logs, and identity signals - that consistently delivered rich context and accurate detections. Today, we’re excited to announce a major expansion: Sentinel UEBA now supports six new data sources including Microsoft first- and third-party platforms like Azure, AWS, GCP, and Okta, bringing deeper visibility, broader context, and more powerful anomaly detection tailored to your environment. This isn’t just about ingesting more logs. It’s about transforming how SOCs understand behavior, detect threats, and prioritize response. With this evolution, analysts gain a unified, cross-platform view of user and entity behavior, enabling them to correlate signals, uncover hidden risks, and act faster with greater confidence. Newly supported data sources are built for real-world security use cases: Authentication activities MDE DeviceLogonEvents – Ideal for spotting lateral movement and unusual access. AADManagedIdentitySignInLogs – Critical for spotting stealthy abuse of non - human identities. AADServicePrincipalSignInLogs - Identifying anomalies in service principal usage such as token theft or over - privileged automation. Cloud platforms & identity management AWS CloudTrail Login Events - Surfaces risky AWS account activity based on AWS CloudTrail ConsoleLogin events and logon related attributes. GCP Audit Logs - Failed IAM Access, Captures denied access attempts indicating reconnaissance, brute force, or privilege misuse in GCP. Okta MFA & Auth Security Change Events – Flags MFA challenges, resets, and policy modifications that may reveal MFA fatigue, session hijacking, or policy tampering. Currently supports the Okta_CL table (unified Okta connector support coming soon). These sources feed directly into UEBA’s entity profiles and baselines - enriching users, devices, and service identities with behavioral context and anomalies that would otherwise be fragmented across platforms. This will complement our existing supported log sources - monitoring Entra ID sign-in logs, Azure Activity logs and Windows Security Events. Due to the unified schema available across data sources, UEBA enables feature-rich investigation and the capability to correlate across data sources, cross platform identities or devices insights, anomalies, and more. AI-powered UEBA that understands your environment Microsoft Sentinel UEBA goes beyond simple log collection - it continuously learns from your environment. By applying AI models trained on your organization’s behavioral data, UEBA builds dynamic baselines and peer groups, enabling it to spot truly anomalous activity. UBEA builds baselines from 10 days (for uncommon activities) to 6 months, both for the user and their dynamically calculated peers. Then, insights are surfaced on the activities and logs - such as an uncommon activity or first-time activity - not only for the user but among peers. Those insights are used by an advanced AI model to identify high confidence anomalies. So, if a user signs in for the first time from an uncommon location, a common pattern in the environment due to reliance on global vendors, for example, then this will not be identified as an anomaly, keeping the noise down. However, in a tightly controlled environment, this same behavior can be an indication of an attack and will surface in the Anomalies table. Including those signals in custom detections can help affect the severity of an alert. So, while logic is maintained, the SOC is focused on the right priorities. How to use UEBA for maximum impact Security teams can leverage UEBA in several key ways. All the examples below leverage UEBA’s dynamic behavioral baselines looking back up to 6 months. Teams can also leverage the hunting queries from the "UEBA essentials" solution in Microsoft Sentinel's Content Hub. Behavior Analytics: Detect unusual logon times, MFA fatigue, or service principal misuse across hybrid environments. Get visibility into geo-location of events and Threat Intelligence insights. Here’s an example of how you can easily discover Accounts authenticating without MFA and from uncommonly connected countries using UEBA behaviorAnalytics table: BehaviorAnalytics | where TimeGenerated > ago(7d) | where EventSource == "AwsConsoleSignIn" | where ActionType == "ConsoleLogin" and ActivityType == "signin.amazonaws.com" | where ActivityInsights.IsMfaUsed == "No" | where ActivityInsights.CountryUncommonlyConnectedFromInTenant == True | evaluate bag_unpack(UsersInsights, "AWS_") | where InvestigationPriority > 0 // Filter noise - uncomment if you want to see low fidelity noise | project TimeGenerated, _WorkspaceId, ActionType, ActivityType, InvestigationPriority, SourceIPAddress, SourceIPLocation, AWS_UserIdentityType, AWS_UserIdentityAccountId, AWS_UserIdentityArn Anomaly detection Identify lateral movement, dormant account reactivation, or brute-force attempts, even when they span cloud platforms. Below are examples of how to discover UEBA Anomalous AwsCloudTrail anomalies via various UEBA activity insights or device insights attributes: Anomalies | where AnomalyTemplateName in ( "UEBA Anomalous Logon in AwsCloudTrail", // AWS ClousTrail anomalies "UEBA Anomalous MFA Failures in Okta_CL", "UEBA Anomalous Activity in Okta_CL", // Okta Anomalies "UEBA Anomalous Activity in GCP Audit Logs", // GCP Failed IAM access anomalies "UEBA Anomalous Authentication" // For Authentication related anomalies ) | project TimeGenerated, _WorkspaceId, AnomalyTemplateName, AnomalyScore, Description, AnomalyDetails, ActivityInsights, DeviceInsights, UserInsights, Tactics, Techniques Alert optimization Use UEBA signals to dynamically adjust alert severity in custom detections—turning noisy alerts into high-fidelity detections. The example below shows all the users with anomalous sign in patterns based on UEBA. Joining the results with any of the AWS alerts with same AWS identity will increase fidelity. BehaviorAnalytics | where TimeGenerated > ago(7d) | where EventSource == "AwsConsoleSignIn" | where ActionType == "ConsoleLogin" and ActivityType == "signin.amazonaws.com" | where ActivityInsights.FirstTimeConnectionViaISPInTenant == True or ActivityInsights.FirstTimeUserConnectedFromCountry == True | evaluate bag_unpack(UsersInsights, "AWS_") | where InvestigationPriority > 0 // Filter noise - uncomment if you want to see low fidelity noise | project TimeGenerated, _WorkspaceId, ActionType, ActivityType, InvestigationPriority, SourceIPAddress, SourceIPLocation, AWS_UserIdentityType, AWS_UserIdentityAccountId, AWS_UserIdentityArn, ActivityInsights | evaluate bag_unpack(ActivityInsights) Another example shows anomalous key vault access from service principal with uncommon source country location. Joining this activity with other alerts from the same service principle increases fidelity of the alerts. You can also join the anomaly UEBA Anomalous Authentication with other alerts from the same identity to bring the full power of UEBA into your detections. BehaviorAnalytics | where TimeGenerated > ago(1d) | where EventSource == "Authentication" and SourceSystem == "AAD" | evaluate bag_unpack(ActivityInsights) | where LogonMethod == "Service Principal" and Resource == "Azure Key Vault" | where ActionUncommonlyPerformedByUser == "True" and CountryUncommonlyConnectedFromByUser == "True" | where InvestigationPriority > 0 Final thoughts This release marks a new chapter for Sentinel UEBA—bringing together AI, behavioral analytics, and cross-cloud and identity management visibility to help defenders stay ahead of threats. If you haven’t explored UEBA yet, now’s the time. Enable it in your workspace settings and don’t forget to enable anomalies as well (in Anomalies settings). And if you’re already using it, these new sources will help you unlock even more value. Stay tuned for our upcoming Ninja show and webinar (register at aka.ms/secwebinars), where we’ll dive deeper into use cases. Until then, explore the new sources, use the UEBA workbook, update your watchlists, and let UEBA do the heavy lifting. UEBA onboarding and setting documentation Identify threats using UEBA UEBA enrichments and insights reference UEBA anomalies referenceMicrosoft Sentinel for SAP Agentless connector GA
Dear Community, Today is the day: Our new agentless connector for Microsoft Sentinel Solution for SAP applications is Generally Available now! Fully onboarded to SAP’s official Business Accelerator Hub and ready for prime time wherever your SAP systems are waiting – on-premises, hyperscalers, RISE, or GROW – to be protected. Let’s hear from an agentless customer: “With the Microsoft Sentinel Solution for SAP and its new agentless connector, we accelerated deployment across our SAP landscape without the complexity of containerized agents. This streamlined approach elevated our SOC’s visibility into SAP security events, strengthened our compliance posture, and enabled faster, more informed incident response” SOC Specialist, North American aviation company Use the video below to kick off your own agentless deployment today. #Kudos to the amazing mvigilante for showing us around the new connector! But we didn’t stop there! Security is being reengineered for the AI era - moving from static, rule-based controls to platform-driven, machine-speed defence that anticipates threats before they strike. Attackers think in graphs - Microsoft does too. We’re bringing relationship-aware context to Microsoft Security - so defenders and AI can see connections, understand the impact of a potential compromise (blast radius), and act faster across pre-breach and post-breach scenarios including SAP systems - your crown jewels. See it in action in below phishing-compromise which lead to an SAP login bypassing MFA with followed operating-system activities on the SAP host downloading trojan software. Enjoy this clickable experience for more details on the scenario. Shows how a phishing compromise escalated to an SAP MFA bypass, highlighting cross-domain correlation. The Sentinel Solution for SAP has AI-first in mind and directly integrates with our security platform on the Defender portal for enterprise-wide signal correlation, Security Copilot reasoning, and Sentinel Data Lake usage. Your real-time SAP detections operate on the Analytics tier for instant results and threat hunting, while the same SAP logs get mirrored to the lake for cost-efficient long-term storage (up to 12 years). Access that data for compliance reporting or historic analysis through KQL jobs on the lake. No more – yeah, I have the data stored somewhere to tick the audit report check box – but be able to query and use your SAP telemetry in long term storage at scale. Learn more here. Findings from the Agentless Connector preview During our preview we learned that majority of customers immediately profit from the far smoother onboarding experience compared to the Docker-based approach. Deployment efforts and time to first SAP log arrival in Sentinel went from days and weeks to hours. ⚠️ Deprecation notice for containerized data connector agent ⚠️ The containerised SAP data connector will be deprecated on 30 September 2026. This change aligns with the discontinuation of the SAP RFC SDK, SAP's strategic integration roadmap, and customer demand for simpler integration. Migrate to the new agentless connector for simplified onboarding and compliance with SAP’s roadmap. All new deployments starting October 31, 2025, will only have the new agentless connector option, and existing customers should plan their migration using the guidance on Microsoft Learn. It will be billed at the same price as the containerized agent, ensuring no cost impact for customers. Note📌: To support transition for those of you on the Docker-based data connector, we have enhanced our built-in KQL functions for SAP to work across data sources for hybrid and parallel execution. Spotlight on new Features Inspired by the feedback of early adopters we are shipping two of the most requested new capabilities with GA right away. Customizable polling frequency: Balance threat detection value (1min intervals best value) with utilization of SAP Integration Suite resources based on your needs. ⚠️Warning! Increasing the intervals may result in message processing truncation to avoid SAP CPI saturation. See this blog for more insights. Refer to the max-rows parameter and SAP documentation to make informed decisions. Customizable API endpoint path suffix: Flexible endpoints allow running all your SAP security integration flows from the agentless connector and adherence to your naming strategies. Furthermore, you can add the community extensions like SAP S/4HANA Cloud public edition (GROW), the SAP Table Reader, and more. Displays the simplified onboarding flow for the agentless SAP connector You want more? Here is your chance to share additional feature requests to influence our backlog. We would like to hear from you! Getting Started with Agentless The new agentless connector automatically appears in your environment – make sure to upgrade to the latest version 3.4.05 or higher. Sentinel Content Hub View: Highlights the agentless SAP connector tile in Microsoft Defender portal, ready for one-click deployment and integration with your security platform The deployment experience on Sentinel is fully automatic with a single button click: It creates the Azure Data Collection Endpoint (DCE), Data Collection Rule (DCR), and Microsoft Entra ID app registration assigned with RBAC role "Monitoring Metrics Publisher" on the DCR to allow SAP log ingest. Explore partner add-ons that build on top of agentless The ISV partner ecosystem for the Microsoft Sentinel Solution for SAP is growing to tailor the agentless offering even further. The current cohort has flagship providers like our co-engineering partner SAP SE themselves with their security products SAP LogServ & SAP Enterprise Threat Detection (ETD), and our mutual partners Onapsis and SecurityBridge. Ready to go agentless? ➤ Get started from here ➤ Explore partner add-ons here. ➤ Share feature requests here. Next Steps Once deployed, I recommend to check AryaG’s insightful blog series for details on how to move to production with the built-in SAP content of agentless. Looking to expand protection to SAP Business Technology Platform? Here you go. #Kudos to the amazing Sentinel for SAP team and our incredible community contributors! That's a wrap 🎬. Remember: bringing SAP under the protection of your central SIEM isn't just a checkbox - it's essential for comprehensive security and compliance across your entire IT estate. Cheers, MartinOperationalizing the Sentinel data lake: A Practitioner’s Guide
This article is part of The Sentinel data lake Practitioner Series. Part 1 of the series focuses on operationalizing the Sentinel data lake and our strategic vision for the customers. This series is evolving based on inputs and feedback from the community as well as various components of turning raw security data and workflows into operational security engine. Why This Series? Microsoft recently announced Sentinel data lake unlocking massive potential for security teams. Security data lakes are the foundation of modern detection and investigation. This blog series is designed to empower you to fully leverage your Sentinel data lake investment – providing practical tools, actionable workflows, and analyst-ready templates that simplify querying datalake-tier data and enable SOC teams to turn raw logs into meaningful security insights. With the right guidance, you can maximize the value you get from your Sentinel data lake. Microsoft Security research team has worked extensively on modular Jupyter notebooks, Python-based data analysis, enrichment, and visualization libraries, and security-driven analysis workflows at scale. We believe the key to adoption lies in researcher-driven operationalization—bringing these methods directly to practitioners in ways they can use immediately. Strategic Vision for Operationalization of Sentinel data lake Our approach centers on researcher-led enablement with ready-to-use workflows and customer community activation. The above infographic outlines four building blocks that brings a security data lake to life: Research curated and Community-Powered Content Hub Researcher-curated GitHub repository. Shared notebooks, detection templates, and models. Continuous contributions from the security researchers and community. Notebook & Model Templates Jupyter & VS Code notebooks tailored for analyst use. ML/GenAI models tailored for security data enrichment and anomaly detection. Modular queries for detections and investigations. Historical Data Enablement Analytics to data lake tier automation for cost-efficient historical queries. Dynamic baselining over months/years of logs to tune detections. Unlocking long-tail investigation scenarios otherwise left dormant. Practical real world Use Cases Historical threat hunting on network, identity, and cloud logs. Dynamic detection tuning at scale. GenAI-powered investigations. Post-incident deep dives to uncover the full blast radius. Getting Started Notebook: Building Familiarity with the Data Lake Framework Before diving into advanced workflows, we’ve published a Getting Started Notebook designed to help practitioners quickly onboard to the Sentinel Data Lake environment. This notebook introduces foundational concepts that will be used across subsequent examples and pipelines. What it covers: Connecting to the Data Lake: Learn how to establish authenticated Spark sessions and securely read data from the Sentinel Data Lake workspace. Exploring Data with Apache Spark: A short hands-on tour using PySpark to inspect schema, preview records, and perform lightweight data transformations at scale. Writing Back to the Lake: Understand the pattern of persisting processed or enriched datasets back to data lake tier for reuse in analytic notebooks and downstream detection pipelines via elevating them to analytics tier. Running Modular Pipelines: Step through a simple example of how pipeline jobs ingest raw security logs (e.g., SigninLogs), apply filters and enrichments, and output ready-to-use tables for later detection development. This foundational notebook ensures analysts and engineers are comfortable with the basic Spark + Sentinel data lake interaction model — the same model used in the advanced operational notebooks (for example, Password Spray Detection or Anomaly Detection workflows) later in this series. Our Commitment This new blog series will serve as a practitioner’s guide for operationalizing security data lakes. In the following weeks, we’ll gradually deliver: Modular Notebook templates to accelerate hunting, baselining, and investigations. End-to-end workflows connecting datalake-tier → analytics-tier → Sentinel detections. Enrichment and Gen AI-driven tools to reduce repetitive manual work and investigation friction. Reusable examples and walkthroughs based on real-world high-volume data sources Our goal is to make Sentinel data lake practical for customers by delivering actionable notebooks, workflows, and enablement. Expected Outcomes for Customers By operationalizing the Sentinel data lake in this way, enterprise customers can expect: Reduced Time-to-Value – Analysts can move from raw logs to actionable detections in days, not months. Improved Detection Quality – Long-term baselining and historical analysis reduce false positives and increase fidelity in your detections. Operational Efficiency – Automated enrichment and packaged workflows minimize manual investigation effort. Cost Optimization – analytics tier -to-data lake tier data workflows avoid expensive, ad-hoc queries and make historical data practical to use. Join the Journey This series is built by practitioners, for practitioners. Alongside blogs, we’ll also share: GitHub repository with reusable notebooks and model templates. Webinars and demos to walk through the workflows. Together, we’ll move beyond storage and make the security data lake truly operational, analyst-friendly, and impactful. Upcoming articles will demonstrate how notebooks and templates can turn research into workflows that are ready for analysts, featuring practical notebook examples available on GitHub. What's next? Join us at Microsoft Ignite in San Francisco on November 17–21, or online, November 18–20, for deep dives and practical labs to help you maximize your Microsoft Defender investments and to get more from the Microsoft capabilities you already use. Security is a core focus at Ignite this year, with the Security Forum on November 17th, deep dive technical sessions, theater talks, and hands-on labs designed for security leaders and practitioners Featured sessions BRK237: Identity Under Siege: Modern ITDR from Microsoft Join experts in Identity and Security to hear how Microsoft is streamlining collaboration across teams and helping customers better protect, detect, and respond to threats targeting your identity fabric. BRK240 – Endpoint security in the AI era: What's new in Defender Discover how Microsoft Defender’s AI-powered endpoint security empowers you to do more, better, faster. BRK236 – Your SOC’s ally against cyber threats, Microsoft Defender Experts See how Defender Experts detect, halt, and manage threats for you, with real-world outcomes and demos. LAB541 – Defend against threats with Microsoft Defender Get hands-on with Defender for Office 365 and Defender for Endpoint, from onboarding devices to advanced attack mitigation. Explore and filter the full security catalog by topic, format, and role: aka.ms/SessionCatalogSecurity. Why attend? Ignite is the place to learn about the latest Defender capabilities, including new agentic AI integrations and unified threat protection. We will also share future-facing innovations in Defender, as part of our ongoing commitment to autonomous defense. Security Forum—Make day 0 count (November 17) Kick off with an immersive, in person preday focused on strategic security discussions and real-world guidance from Microsoft leaders and industry experts. Select Security Forum during registration. Register for Microsoft Ignite >Automating IOC hunts in Microsoft Sentinel data lake
Security operations are undergoing significant transformation driven by the introduction of AI and a rapidly evolving threat landscape. With Microsoft Sentinel data lake now generally available, organizations can centralize all their security data in a purpose-built security data lake. This helps optimize costs, simplify data management, and accelerate the adoption of AI in security operations. This empowers defenders to transcend legacy security controls, adopting advanced analytics and automation for more dynamic and effective protection. A key advantage of the Sentinel data lake is its cost-efficiency, making it ideal for ingesting and retaining large volumes of security logs, such as network logs, without incurring high expenses or compromising coverage. By storing all security data in a unified, cost-effective data lake, organizations gain comprehensive, long-term visibility for historical threat hunting and TI matching, enabling investigations across extended timelines without the prohibitive costs of traditional analytics solutions. In this blog we will explore how security teams can leverage KQL jobs in Sentinel data lake to automate threat hunting and threat intelligence matching across network logs, enabling scalable, cost-effective, and continuous threat detection. By doing so, SOCs can efficiently process large volumes of data and transform raw logs into actionable insights efficiently with minimal manual intervention. What are KQL jobs? KQL jobs in Sentinel data lake are automated one-time or scheduled jobs that run Kusto Query Language (KQL) queries on data lake. These jobs help security teams investigate and hunt for threats more easily by automating processes like checking logs against known threat data. By automating tasks such as IOC matching with historical or high-volume data, analysts are able to concentrate on higher-value activities. This results in more effective threat detection and response. The next section demonstrates how to use the data lake for Threat Intelligence (TI) matching across network logs. IOC matching on network log on data lake Network logs, such as firewall and proxy data, are essential for uncovering advanced threats and supporting investigations. However, storing all this data in the analytics tier is often expensive, leading to reduced retention and potential blind spots. With Sentinel data lake, SOCs can store all their raw telemetry, at a fraction of the cost, making it possible to hunt for threats across a much broader timeline without financial constraints. However, simply storing data isn’t enough. To turn raw logs into actionable insights, SOC teams need to automate both summarization and threat intelligence (TI) matching. Scheduled KQL jobs make this possible by scanning new data in a schedule as it arrives in the data lake, surfacing suspicious activity for analyst review. Schedule KQL job for TI matching on network logs Here’s a practical example of how a SOC can use a scheduled KQL job to summarize network activity and correlate it with threat intelligence indicators. In this scenario, a KQL job is run to identify network log entries from Palo Alto firewalls that match known malicious IPs from ThreatIntelIndicators table. The output provides the complete network log row, enriched with relevant threat intelligence fields for further investigation and response. Create your query: let IPRegex = '[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}'; let dt_lookBack = 75m; // Look back 1 hour for CommonSecurityLog events let ioc_lookBack = 14d; // Look back 14 days for threat intelligence indicators // Fetch threat intelligence indicators related to IP addresses let IP_Indicators = ThreatIntelIndicators //extract key part of kv pair | extend IndicatorType = replace(@"\[|\]|\""", "", tostring(split(ObservableKey, ":", 0))) | where IndicatorType in ("ipv4-addr", "ipv6-addr", "network-traffic") | extend NetworkSourceIP = toupper(ObservableValue) | extend TrafficLightProtocolLevel = tostring(parse_json(AdditionalFields).TLPLevel) | where TimeGenerated >= ago(ioc_lookBack) | extend TI_ipEntity = iff(isnotempty(NetworkSourceIP), NetworkSourceIP, NetworkSourceIP) | extend TI_ipEntity = iff(isempty(TI_ipEntity) and isnotempty(NetworkSourceIP), NetworkSourceIP, TI_ipEntity) | where ipv4_is_private(TI_ipEntity) == false and TI_ipEntity !startswith "fe80" and TI_ipEntity !startswith "::" and TI_ipEntity !startswith "127." | summarize LatestIndicatorTime = arg_max(TimeGenerated, *) by Id, ObservableValue | where IsActive and (ValidUntil > now() or isempty(ValidUntil)); // Perform a join between IP indicators and CommonSecurityLog events IP_Indicators | project-reorder *, Tags, TrafficLightProtocolLevel, NetworkSourceIP, TI_ipEntity // Use innerunique to keep performance fast and result set low, as we only need one match to indicate potential malicious activity that needs investigation | join kind=innerunique ( CommonSecurityLog | where TimeGenerated >= ago(dt_lookBack) | extend MessageIP = extract(IPRegex, 0, Message) | extend CS_ipEntity = iff((not(ipv4_is_private(SourceIP)) and isnotempty(SourceIP)), SourceIP, DestinationIP) | extend CS_ipEntity = iff(isempty(CS_ipEntity) and isnotempty(MessageIP), MessageIP, CS_ipEntity) | extend CommonSecurityLog_TimeGenerated = TimeGenerated ) on $left.TI_ipEntity == $right.CS_ipEntity // Filter out logs that occurred after the expiration of the corresponding indicator | where CommonSecurityLog_TimeGenerated < ValidUntil // Group the results by IndicatorId and CS_ipEntity, and keep the log entry with the latest timestamp | summarize CommonSecurityLog_TimeGenerated = arg_max(CommonSecurityLog_TimeGenerated, *) by Id, CS_ipEntity // Select the desired output fields | project timestamp = CommonSecurityLog_TimeGenerated, SourceIP, DestinationIP, MessageIP, Message, DeviceVendor, DeviceProduct, Id, ValidUntil, Confidence, TI_ipEntity, CS_ipEntity, LogSeverity, DeviceAction Source: Microsoft Sentinel GitHub repo Before submitting a KQL job you may want to test your query interactively, using the KQL queries page: Create a KQL job: To match against new logs periodically, we would like to schedule this job to run every hour to summarize network log and match against latest IOCs in ThreatInelIndicators. To avoid missing any logs, I suggest adding an overlap between lookback and schedules, to make sure all logs are scanned. For example, you can set lookback of the last 75 minutes and execute job runs every 60 minutes. KQL jobs can run ad-hoc or be scheduled based on your preferred frequency (by minutes, hourly, daily, weekly or monthly), automatically summarizing new network activity and highlighting matches with known malicious indicators. Analysts can then focus on the most relevant events, accelerating investigations and reducing noise. Results are automatically available in the analytics tier and can be used to set up an automated detection using Analytics rules. The cost of running KQL jobs in Sentinel data lake depends on the volume of data scanned and how frequently the jobs run. Data lake KQL queries and jobs are priced at $0.005 per GB scanned. For example, if a KQL job scans 1 TB of data daily, the monthly cost would be around $150 USD. This pricing model allows organizations to perform large-scale threat hunting and intelligence matching without the high expenses typically associated with traditional SIEMs. $0.005 per GB scanned. For more details around Microsoft Sentinel data lake costs for KQL queries and jobs, see https://azure.microsoft.com/en-us/pricing/calculator. Summary and next steps Threat hunting at scale within Sentinel data lake is simplified with KQL jobs. SOC teams can use this method for various hunting or anomaly detection scenarios such as efficiently aggregating and correlating network logs with threat intelligence, enhancing visibility, agility, and assurance, and transforming raw telemetry into actionable security insights. KQL jobs provide several benefits: Continuous threat coverage: Scheduled jobs utilizing KQL automatically correlate high-volume logs located directly in the data lake with up-to-date threat intelligence. This process helps minimize detection gaps and blind spots. Efficient use of resources: Automating TI matching saves analysts from repetitive queries, allowing them to focus on investigating validated alerts rather than sifting through raw logs. Faster response times: Suspicious connections flagged by minutes or every hour enable quicker triage and containment before threats escalate. Historical context: Matches are retained against long-term or high volume logs, enabling analysts to trace back patterns of malicious activity and support deeper investigations. Get started with Microsoft Sentinel data lake today. Microsoft Sentinel data lake overview - Microsoft Security | Microsoft Learn KQL and the Microsoft Sentinel data lake - Microsoft Security | Microsoft Learn Microsoft Sentinel Pricing | Microsoft Security What's next? Join us at Microsoft Ignite in San Francisco on November 17–21, or online, November 18–20, for deep dives and practical labs to help you maximize your Microsoft Defender investments and to get more from the Microsoft capabilities you already use. Security is a core focus at Ignite this year, with the Security Forum on November 17th, deep dive technical sessions, theater talks, and hands-on labs designed for security leaders and practitioners Featured sessions BRK237: Identity Under Siege: Modern ITDR from Microsoft Join experts in Identity and Security to hear how Microsoft is streamlining collaboration across teams and helping customers better protect, detect, and respond to threats targeting your identity fabric. BRK240 – Endpoint security in the AI era: What's new in Defender Discover how Microsoft Defender’s AI-powered endpoint security empowers you to do more, better, faster. BRK236 – Your SOC’s ally against cyber threats, Microsoft Defender Experts See how Defender Experts detect, halt, and manage threats for you, with real-world outcomes and demos. LAB541 – Defend against threats with Microsoft Defender Get hands-on with Defender for Office 365 and Defender for Endpoint, from onboarding devices to advanced attack mitigation. Explore and filter the full security catalog by topic, format, and role: aka.ms/SessionCatalogSecurity. Why attend? Ignite is the place to learn about the latest Defender capabilities, including new agentic AI integrations and unified threat protection. We will also share future-facing innovations in Defender, as part of our ongoing commitment to autonomous defense. Security Forum—Make day 0 count (November 17) Kick off with an immersive, in person preday focused on strategic security discussions and real-world guidance from Microsoft leaders and industry experts. Select Security Forum during registration. Register for Microsoft Ignite >
What’s New in Microsoft Sentinel: November 2025
Welcome to our new Microsoft Sentinel blog series! We’re excited to launch a new blog series focused on Microsoft Sentinel. From the latest product innovations and feature updates to industry recognition, success stories, and major events, you’ll find it all here. This first post kicks off the series by celebrating Microsoft’s recognition as a Leader in the 2025 Gartner Magic Quadrant for SIEM 1 . It also introduces the latest innovations designed to deliver measurable impact and empower defenders with adaptable, collaborative tools in an evolving threat landscape. Microsoft is recognized as a Leader in 2025 Gartner Magic Quadrant for Security Information and Event Management (SIEM) Microsoft Sentinel continues to drive security innovation—and the industry is taking notice. Microsoft was named a leader in the 2025 Gartner Magic Quadrant for Security Information and Event Management (SIEM) 1 , published on October 8, 2025. We believe this acknowledgment reinforces our commitment to helping organizations stay secure in a rapidly changing threat landscape. Read blog for more information. Take advantage of M365 E5 benefit and Microsoft Sentinel promotional pricing Microsoft 365 E5 benefit Customers with Microsoft 365 E5, A5, F5, or G5 licenses automatically receive up to 5 MB of free data ingestion per user per day, covering key security data sources like Azure AD sign-in logs and Microsoft Cloud App Security discovery logs—no enrollment required. Read more about M365 benefits for Microsoft Sentinel. New 50GB promotional pricing To make Microsoft Sentinel more accessible to small and mid-sized organizations, we introduced a new 50 GB commitment tier in public preview, with promotional pricing starting October 1, 2025, through March 31, 2026. Customers who choose the 50 GB commitment tier during this period will maintain their promotional rate until March 31, 2027. Available globally with regional variations in regional pricing it is accessible through EA, CSP, and Direct channels. For more information see Microsoft Sentinel pricing page. Partner Integrations: Strengthening TI collaboration and workflow automation Microsoft Sentinel continues to expand its ecosystem with powerful partner integrations that enhance security operations. With Cyware, customers can now share threat intelligence bi-directionally across trusted destinations, ISACs, and multi-tenant environments—enabling real-time intelligence exchange that strengthens defenses and accelerates coordinated response. Learn more about the Cyware integration. Learn more about the Cyware integration here. Meanwhile, BlinkOps integration combined with Sentinel’s SOAR capabilities empowers SOC teams to automate repetitive tasks, orchestrate complex playbooks, and streamline workflows end-to-end. This automation reduces operational overhead, cuts Mean Time to Respond (MTTR) and frees analysts for strategic threat hunting. Learn more about the BlinkOps integration. Learn more about the BlinkOps integration. Harnessing Microsoft Sentinel Innovations Security is being reengineered for the AI era, moving beyond static, rule-based controls and reactive post-breach response toward platform-led, machine-speed defense. To overcome fragmented tools, sprawling signals, and legacy architectures that cannot keep pace with modern attacks, Microsoft Sentinel has evolved into both a SIEM and a unified security platform for agentic defense. These updates introduce architectural enhancements and advanced capabilities that enable AI-driven security operations at scale, helping organizations detect, investigate, and respond with unprecedented speed and precision. Microsoft Sentinel graph – Public Preview Unified graph analytics for deeper context and threat reasoning. Microsoft Sentinel graph delivers an interactive, visual map of entity relationships, helping analysts uncover hidden attack paths, lateral movement, and root causes for pre- and post-breach investigations. Read tech community blog for more details. Microsoft Sentinel Model Context Protocol (MCP) server – Public Preview Context is key to effective security automation. Microsoft Sentinel MCP server introduces a standardized protocol for building context-aware solutions, enabling developers to create smarter integrations and workflows within Sentinel. This opens the door to richer automation scenarios and more adaptive security operations. Read tech community blog for more details. Enhanced UEBA with New Data Sources – Public Preview We are excited to announce support for six new sources in our user entity and behavior analytics algorithm, including AWS, GCP, Okta, and Azure. Now, customers can gain deeper, cross-platform visibility into anomalous behavior for earlier and more confident detection. Read our blog and check out our Ninja Training to learn more. Developer Solutions for Microsoft Sentinel platform – Public Preview Expanded APIs, solution templates, and integration capabilities empower developers to build and distribute custom workflows and apps via Microsoft Security Store. This unlocks faster innovation, streamlined operations, and new revenue opportunities, extending Sentinel beyond out-of-the-box functionality for greater agility and resilience. Read tech community blog for more details. Growing ecosystem of Microsoft Sentinel data connectors We are excited to announce the general availability of four new data connectors: AWS Server Access Logs, Google Kubernetes Engine, Palo Alto CSPM, and Palo Alto Cortex Xpanse. Visit find your Microsoft Sentinel data connector page for the list of data connectors currently supported. We are also inviting Private Previews for four additional connectors: AWS EKS, Qualys VM KB, Alibaba Cloud Network, and Holm Security towards our commitment to expand the breadth and depth to support new data sources. Our customer support team can help you sign up for previews. New agentless data connector for Microsoft Sentinel Solution for SAP applications We’re excited to announce the general availability of a new agentless connector for Microsoft Sentinel solution for SAP applications, designed to simplify integration and enhance security visibility. This connector enables seamless ingestion of SAP logs and telemetry directly into Microsoft Sentinel, helping SOC teams monitor critical business processes, detect anomalies, and respond to threats faster—all while reducing operational overhead. Events, Webinars and Training Stay connected with the latest security innovation and best practices. From global conferences to expert-led sessions, these events offer opportunities to learn, network, and explore how Microsoft is shaping AI-driven, end-to-end security for the modern enterprise. Microsoft Ignite 2025 Security takes center stage at Microsoft Ignite, with dedicated sessions and hands-on experiences for security professionals and leaders. Join us in San Francisco, November 17–21, 2025, or online, to explore our AI-first, end-to-end security platform designed to protect identities, devices, data, applications, clouds, infrastructure—and critically—AI systems and agents. Register today! Microsoft Security Webinars Stay ahead of emerging threats and best practices with expert-led webinars from the Microsoft Security Community. Discover upcoming sessions on Microsoft Sentinel SIEM & platform, Defender, Intune, and more. Sign up today and be part of the conversation that shapes security for everyone. Learn more about upcoming webinars. Onboard Microsoft Sentinel in Defender – Video Series Microsoft leads the industry in both SIEM and XDR, delivering a unified experience that brings these capabilities together seamlessly in the Microsoft Defender portal. This integration empowers security teams to correlate insights, streamline workflows, and strengthen defenses across the entire threat landscape. Ready to get started? Explore our video series to learn how to onboard your Microsoft Sentinel experience and unlock the full potential of integrated security. Watch Microsoft Sentinel is now in Defender video series. MDTI Convergence into Microsoft Sentinel & Defender XDR overview Discover how Microsoft Defender Threat Intelligence Premium is transforming cybersecurity by integrating into Defender XDR, Sentinel, and the Defender portal. Watch this session to learn about new features, expanded access to threat intelligence, and how these updates strengthen your security posture. Partner Sentinel Bootcamp Transform your security team from Sentinel beginners to advanced practitioners. This comprehensive 2-day bootcamp helps participants master architecture design, data ingestion strategies, multi-tenant management, and advanced analytics while learning to leverage Microsoft's AI-first security platform for real-world threat detection and response. Register here for the bootcamp. Looking to dive deeper into Microsoft Sentinel development? Check out the official https://aka.ms/AppAssure_SentinelDeveloper. It’s the central reference for developers and security teams who want to build custom integrations, automate workflows, and extend Sentinel’s capabilities. Bookmark this link as your starting point for hands-on guidance and tools. Stay Connected Check back each month for the latest innovations, updates, and events to ensure you’re getting the most out of Microsoft Sentinel. 1 Gartner® Magic Quadrant™ for Security Information and Event Management, Andrew Davies, Eric Ahlm, Angel Berrios, Darren Livingstone, 8 October 20256 truths about migrating Microsoft Sentinel to the Defender portal
The move from the Azure portal to the Microsoft Defender portal is one of the most significant transformations yet for Microsoft Sentinel SIEM. By July 1, 2026, every Sentinel environment will make this leap. But this isn’t just a new coat of paint—it’s a re-architecture of how Security Operations Centers (SOCs) detect, investigate, and respond to threats. For many teams, the shift will feel different, even surprising. But within those surprises lie opportunities to run leaner, smarter, and more resilient operations. Here are six insights that will help you prepare your SOC to not only manage the change but thrive in it. The migration landscape: what's at stake This comprehensive guide distills the six most impactful changes every SOC needs to understand before making the switch. The migration represents a fundamental paradigm shift in how security operations are conducted, moving from the familiar Azure portal environment to the Defender portal—delivering new capabilities such as enhanced correlation, streamlined workflows, and continued innovation. The stakes are high. With this shift, SOC teams must prepare not just for new interfaces, but for entirely new ways of thinking about incident management, automation, and security operations. To thrive in this new experience, organizations must recognize this migration as an opportunity to modernize their security operations, while those that treat it as a simple interface change may find themselves struggling with unexpected disruptions. Critical Timeline: All Microsoft Sentinel environments must migrate to the Defender portal by July 1, 2026. This is not optional. Truth #1: You will gain a more comprehensive view of incidents The Defender correlation engine automatically groups related alerts into a single incident and merges existing incidents it deems to be sufficiently alike based on shared entities, attack patterns, and timing. This process is designed to reduce noise and provide a more context-rich view of an attack, greatly benefiting the SOC by transforming scattered, benign signals into a complete incident view. More comprehensive incidents We observed up to an 80% drop in Sentinel incident counts for early adopters of the unified SOC. Fewer duplications When Defender XDR merges one incident into another, the original "source incident" is automatically closed, given a Redirected tag, and disappears from the main queue. A shorter incident queue Analysts will enjoy fewer singleton alerts with the enhanced correlation engine. Leveraging the new correlation engine in Defender allows customers to transform signals into stories. With more comprehensive incidents and fewer singleton alerts, customers can connect the dots across their environment. How the correlation engine decides The Defender correlation engine uses sophisticated algorithms to determine which incidents should be merged. It analyzes multiple factors simultaneously to make these decisions, creating a more comprehensive view of security events. Shared entities (users, devices, IP addresses) Attack patterns and techniques Temporal proximity of events Contextual relationships between alerts Threat intelligence correlations This automated correlation is designed to mirror the thought process of an experienced analyst who would naturally connect related security events. The engine operates at scale and speed that human analysts cannot match, processing thousands of potential correlations simultaneously. Truth #2: Automation will level up Think of this migration as a chance to strengthen the foundation of your Security Orchestration, Automation, and Response (SOAR) strategy. The new unified schema encourages automation that’s more resilient and future-proof. By keying off stable fields like AnalyticsRuleName and tags, SOC teams gain precision and durability in their workflows—ensuring automation continues to work as correlation logic evolves. Some playbooks and automations built in the Azure portal will need adjustment. Fields like IncidentDescription and IncidentProvider behave differently now, and titles aren’t always stable identifiers. Adapting your automation rules and playbooks: The Description field is gone Customers will need to update automations, playbooks and integrations that rely on the SecurityIncident table. This table no longer populates the Description field, which impacts rules that use dynamic content from KQL queries to populate descriptions. IncidentProvider property removed Since all incidents in the unified portal are considered to come from XDR, this property is now obsolete. Automation rules that use a condition like "Incident Provider equals Microsoft Sentinel" will need to be updated or removed entirely. Incident titles are unstable The Defender correlation engine may change an incident's title when it merges alerts or other incidents. Instead, use Analytics rule name or tags. Fixing Automation: The Action Plan Audit existing automation Conduct a comprehensive review of all automation rules, SOAR playbooks, and integration scripts that interact with Sentinel incidents. Document dependencies on deprecated fields and identify potential failure points. Update field references Modify your rules to use more stable identifiers. Replace references to IncidentDescription and IncidentProvider with fields like AnalyticsRuleName and tags for conditions, ensuring more reliable automation execution. Implement granular control Use Analytics rule name as the recommended method for ensuring an automation rule only targets incidents generated from a specific Sentinel rule, preserving granular control over your workflows. Test and validate Thoroughly test updated automation in a staging environment before deployment. Create test scenarios that simulate the new correlation behaviors to ensure automation responds appropriately to merged incidents. Teams must move away from dependencies on dynamic content and toward predictable data points that will remain consistent even as the correlation engine modifies incident properties. Truth #3: The correlation engine is in charge The Defender correlation engine acts like an intelligent filter—connecting the dots between alerts at machine speed. You can’t turn it off, but you can work with it. Some creative SOCs are already finding ways to guide its behavior, like routing test incidents separately to prevent unwanted merges. While it may feel like giving up some manual control, the payoff is big: an analyst’s effort shifts from sorting noise to validating signals. In short, analysts spend less time as traffic cops and more time acting as investigators. Rapid response with less configuration With Defender’s advanced correlation algorithms handling incident grouping automatically, your security teams can confidently dedicate their attention to strategic decision-making and rapid incident response, rather than manual rule configuration. Truth #4: Some familiar tools stay behind, but new ones await A few Sentinel features you’ve grown used to—like Incident Tasks and manually created incidents—don’t carry over. Alerts that aren’t tied to incidents won’t appear in the Defender portal either. It’s a shift, yes. But one that nudges SOCs toward leaner workflows and built-in consistency. Instead of juggling multiple tracking models, analysts can focus on cases that Defender elevates as truly significant. Incident tasks The built-in checklist and workflow management feature used to standardize analyst actions within an incident is unavailable in the Defender portal. Teams must adopt alternative methods such as Cases for ensuring consistent investigation procedures. Alerts without incidents Analytics rules configured only to generate alerts (with createIncident set to false) will still write to the SecurityAlerts table. However, while the Advanced hunting query editor doesn’t recognize the SecurityAlerts table schema, you can still use the table in queries and analytics rules. Manually created incidents While incidents created manually or programmatically in Sentinel do not sync to the Defender portal, they remain fully manageable through API, ensuring specific workflows are handled separately. You can ingest the relevant events as custom logs and then create analytics rules to generate incidents from that data. Similar incidents feature Analysts will have access to a more advanced and integrated correlation engine that automatically merges related incidents from all sources. This built-in engine proactively groups threats by identifying common entities and attack behaviors, presenting a single, comprehensive incident rather than requiring manual investigation of separate alerts. Truth #5: Permissions grow more sophisticated Operating in the Defender portal introduces a dual permissions model: your Azure Role-Based Access Control (RBAC) roles still apply, but you’ll also need Microsoft Entra ID or Defender Unified RBAC for high-level tasks. This layered model means more careful planning up front, but it also opens the door for clearer separation of duties across teams—making it easier to scale SOC operations securely. Additive permission model After connecting to the Defender portal, your existing Azure RBAC permissions are still honored and allow you to work with the Microsoft Sentinel features you already have access to. The new permissions are for accessing the Defender portal itself and performing high-level onboarding actions. Dual permission requirements Analysts and engineers need either global Microsoft Entra ID roles or roles from the new Microsoft Defender XDR Unified RBAC model to access and operate within the Defender portal. Elevated setup requirements Key actions like onboarding a workspace for the first time or changing the primary workspace require either Global Administrator or Security Administrator roles in Microsoft Entra ID. Permission planning matrix User type Required permissions Access capabilities SOC Analyst Defender XDR role + existing Sentinel RBAC Incident investigation, alert triage, basic response actions SOC Manager Security Reader/Operator + Defender XDR roles Full operational access, reporting, dashboard configuration Security Engineer Contributor + Defender XDR Administrator Rule creation, automation management, advanced configuration Initial Setup Admin Global Admin or Security Admin Workspace onboarding, primary workspace designation MSSP Technician B2B Guest + appropriate Defender roles Customer tenant access via guest invitation model Planning permission transitions requires mapping current access patterns to the new dual-model requirements. Organizations should audit existing permissions, identify gaps, and develop migration plans that ensure continued access while meeting the new portal requirements. Truth #6: Primary vs. secondary workspace divide Not all workspaces are equal in the new model. The primary workspace is the one fully integrated with Defender . It’s where cross-signal correlation happens, enabling unified incidents that combine alerts across Defender and Microsoft Sentinel for a truly comprehensive view of threats. This forces a big but valuable decision: which workspace will anchor your organization’s most critical signals? Choosing wisely ensures you unlock the full power of cross-signal correlation, giving your SOC a panoramic view of complex threats. Primary workspace This is the only workspace with alerts that can be correlated with Defender alerts and included together in unified incidents. It receives the full benefit of cross-signal correlation and advanced threat detection capabilities. Secondary workspaces Secondary workspaces often exist for compliance, regulatory, or business unit separation reasons. While their alerts are not correlated with XDR or or other workspaces, this design keeps them logically and operationally isolated. Importantly, analysts who need visibility can still be granted URBAC permissions to view incidents and alerts, even if those originate outside their local workspace. Selecting your primary workspace is a strategic step that unlocks the full potential of our unified SIEM and XDR correlation capabilities, empowering you with deeper insights and stronger threat detection across your critical data sources. While secondary workspaces remain fully operational, consolidating workspaces strategically allows your organization to maximize the advanced correlation and proactive defense that define the core value of the unified platform. Thoughtful workspace consolidation, balanced with compliance and cost considerations, positions your organization to get the absolute best out of your security investments. Turning migration into momentum The Defender portal migration isn’t just a deadline to meet—it’s an opportunity to re-engineer how your SOC operates. By anticipating these six shifts, you can move past disruption and toward a strategic advantage: fewer distractions, stronger automation, richer incident context, and a unified XDR-driven defense posture. The future SOC isn’t just managing alerts—it’s mastering context. And the Defender portal is your launchpad. More Information Transition Your Microsoft Sentinel Environment to the Defender Portal | Microsoft Learn Frequently asked questions about the unified security operations platform | Microsoft Community Hub Alert correlation and incident merging in the Microsoft Defender portal - Microsoft Defender XDR | Microsoft Learn Planning your move to Microsoft Defender portal for all Microsoft Sentinel customers | Microsoft Community Hub Connect Microsoft Sentinel to the Microsoft Defender portal - Unified security operations | Microsoft Learn Microsoft Sentinel in the Microsoft Defender portal | Microsoft LearnGo agentless with Microsoft Sentinel Solution for SAP
What a title during Agentic AI times 😂 📢UPDATE: Agentless reached GA! See details here. Dear community, Bringing SAP workloads under the protection of your SIEM solution is a primary concern for many customers out there. The window for defenders is small “Critical SAP vulnerabilities being weaponized in less than 72 hours of a patch release and new unprotected SAP applications provisioned in cloud (IaaS) environments being discovered and compromised in less than three hours.” (SAP SE + Onapsis, Apr 6 2024) Having a turn-key solution as much as possible leads to better adoption of SAP security. Agent-based solutions running in Docker containers, Kubernetes, or other self-hosted environemnts are not for everyone. Microsoft Sentinel for SAP’s latest new capability re-uses the SAP Cloud Connector to profit from already existing setups, established integration processes, and well-understood SAP components. Meet agentless ❌🤖 The new integration path leverages SAP Integration Suite to connect Microsoft Sentinel with your SAP systems. The Cloud integration capability of SAP Integration Suite speaks all relevant protocols, has connectivity into all the places where your SAP systems might live, is strategic for most SAP customers, and is fully SAP RISE compatible by design. Are you deployed on SAP Business Technology Platform yet? Simply upload our Sentinel for SAP integration package (see bottom box in below image) to your SAP Cloud Integration instance, configure it for your environment, and off you go. Best of all: The already existing SAP security content (detections, workbooks, and playbooks) in Microsoft Sentinel continues to function the same way as it does for the Docker-based collector agent variant. The integration marks your steppingstone to bring your SAP threat signals into the Unified Security Operations Platform – a combination of Defender XDR and Sentinel – that looks beyond SAP at your whole IT estate. Microsoft Sentinel solution for SAP applications is certified for SAP S/4HANA Cloud, Private Edition RISE with SAP, and SAP S/4HANA on-premises. So, you are all good to go😎 You are already dockerized or agentless? Then proceed to this post to learn more about what to do once the SAP logs arrived in Sentinel. Final Words During the preview we saw drastically reduced deployment times for SAP customers being less familiar with Docker, Kubernetes and Linux administration. Cherry on the cake: the network challenges don’t have to be tackled again. The colleagues running your SAP Cloud Connector went through that process a long time ago. SAP Basis rocks 🤘 Get started from here on Microsoft Learn. Find more details on our blog on the SAP Community. Cheers MartinRun agentless SAP connector cost-efficiently
The SAP agentless connector uses SAP Integration Suite (Cloud Integration/CPI) to fetch SAP audit log data and forward it to Microsoft Sentinel. Because SAP CPI billing typically reflects message counts and data volume, you can tune the connector to control costs—while preserving reliability and timeliness. Cost reductions primarily come from sending fewer CPI messages by increasing the polling interval. The max-rows parameter is a stability safeguard that caps events per run to protect CPI resources; it is not a direct cost-optimization lever. After any change, monitor CPI execution time and resource usage. ☝️Note: It may not be feasible to increase the polling interval on busy systems processing large data volumes. Larger intervals can lengthen CPI execution time and cause truncation when event spikes exceed max-rows. Cost optimization via longer intervals generally works best on lower-utilization environments (for example, dev and test) where event volume is modest and predictable. Tunable parameters Setting Default Purpose Cost impact Risk / trade-off Polling interval 1 minute How often the connector queries SAP and triggers a CPI message. Lower message count at longer intervals → potential cost reduction. Larger batches per run can extend CPI execution time; spikes may approach max-rows after which message processing for that interval is truncated. max-rows 150,000 Upper bound on events packaged per run to protect CPI stability. None (safeguard)—does not reduce message count on its own. If too low, frequent truncation; if too high, runs may near CPI resource limits. Adjust cautiously and observe. ☝️Note: When event volume within one interval exceeds max-rows, the batch is truncated by design. Remaining events are collected on subsequent runs. Recommended approach Start with defaults. Use a 1-minute polling interval and max-rows = 150,000. Measure your baseline. Understand average and peak ingestion per minute (see KQL below). Optimize the polling interval first to reduce message count when costs are a concern. Treat max-rows as a guardrail. Change only if you consistently hit the cap; increase in small steps. Monitor after each change. Track CPI run duration, CPU/memory, retries/timeouts, and connector health in both SAP CPI and Sentinel. 💡Aim for the lowest interval that keeps CPI runs comfortably within execution-time and resource limits. Change one variable at a time and observe for at least a full business cycle. 🧐Consider the Azure Monitor Log Ingestion API limits to close the loop on your considerations. Analyze ingestion profile (KQL) ABAPAuditLog | where TimeGenerated >= ago(90d) | summarize IngestedEvents = count() by bin(UpdatedOn, 1m) | summarize MaxEvents = max(IngestedEvents), AverageEvents = toint(avg(IngestedEvents)), P95_EventsPerMin = percentile(IngestedEvents, 95) How to use these metrics? AverageEvents → indicates typical per-minute volume. P95_EventsPerMin → size for spikes: choose a polling interval such that P95 × interval (minutes) remains comfortably below max-rows. If MaxEvents × interval approaches max-rows, expect truncation and catch-up behavior—either shorten the interval or, if safe, modestly raise max-rows. Operational guidance ⚠️❗Large jumps (for example, moving from a 1-minute interval to 5 minutes and raising max-rows simultaneously) can cause CPI runs to exceed memory/time limits. Adjust gradually and validate under peak workloads (e.g., period close, audit windows). Document changes (interval, max-rows, timestamp, rationale). Alert on CPI anomalies (timeouts, retries, memory warnings). Re-evaluate regularly in higher-risk periods when SAP event volume increases. Balancing Audit Log Tuning and Compliance in SAP NetWeaver: Risks of Excluding Users and Message Classes When tuning SAP NetWeaver audit logging via transaction SM19 (older releases) or RSAU_CONFIG (newer releases), administrators can filter by user or message class to reduce log volume - such as excluding high-volume batch job users or specific event types - but these exclusions carry compliance risks: omitting audit for certain users or classes may undermine traceability, violate regulatory requirements, or mask unauthorized activities, especially if privileged or technical users are involved. Furthermore, threat hunting in Sentinel for SAP gets "crippled" due to missing insights. Best practice is to start with comprehensive logging, only apply exclusions after a documented risk assessment, and regularly review settings to ensure that all critical actions remain auditable and compliant with internal and external requirements. Cost-Efficient Long-Term Storage for Compliance Microsoft Sentinel Data Lake enables organizations to retain security logs - including SAP audit data - for up to 12 years at a fraction of traditional SIEM storage costs, supporting compliance with regulations such as NIS2, DORA and more. By decoupling storage from compute, Sentinel Data Lake allows massive volumes of security data to be stored cost-effectively in a unified, cloud-native platform, while maintaining full query and analytics capabilities for forensic investigations and regulatory reporting. This approach ensures that organizations can meet strict data retention and auditability requirements without compromising on cost or operational efficiency. Summary Use the polling interval to reduce message count (primary cost lever). Keep max-rows as a safety cap to protect CPI stability. Measure → adjust → monitor to achieve a stable, lower-cost configuration tailored to your SAP workload. Use built-in mirroring to the Sentinel Data Lake to store the SAP audit logs cost-efficient for years Next Steps Explore agentless SAP deployment config guide on Microsoft Learn Expand deployment to SAP Business Technology Platform Use the insightful blog series by AryaG for details on how to move to production with the built-in SAP content of agentlessAutomating Microsoft Sentinel: Part 2: Automate the mundane away
Welcome to the second entry of our blog series on automating Microsoft Sentinel. In this series, we’re showing you how to automate various aspects of Microsoft Sentinel, from simple automation of Sentinel Alerts and Incidents to more complicated response scenarios with multiple moving parts. So far, we’ve covered Part 1: Introduction to Automating Microsoft Sentinel where we talked about why you would want to automate as well as an overview of the different types of automation you can do in Sentinel. Here is a preview of what you can expect in the upcoming posts [we’ll be updating this post with links to new posts as they happen]: Part 1: Introduction to Automating Microsoft Sentinel Part 2: Automation Rules [You are here] – Automate the mundane away Part 3: Playbooks 1 – Playbooks Part I – Fundamentals Part 4: Playbooks 2 – Playbooks Part II – Diving Deeper Part 5: Azure Functions / Custom Code Part 6: Capstone Project (Art of the Possible) – Putting it all together Part 2: Automation Rules – Automate the mundane away Automation rules can be used to automate Sentinel itself. For example, let’s say there is a group of machines that have been classified as business critical and if there is an alert related to those machines, then the incident needs to be assigned to a Tier 3 response team and the severity of the alert needs to be raised to at least “high”. Using an automation rule, you can take one analytic rule, apply it to the entire enterprise, but then have an automation rule that only applies to those business-critical systems to make those changes. That way only the items that need that immediate escalation receive it, quickly and efficiently. Automation Rules In Depth So, now that we know what Automation Rules are, let’s dive in to them a bit deeper to better understand how to configure them and how they work. Creating Automation Rules There are three main places where we can create an Automation Rule: 1) Navigating to Automation under the left menu 2) In an existing Incident via the “Actions” button 3) When writing an Analytic Rule, under the “Automated response” tab The process for each is generally the same, except for the Incident route and we’ll break that down more in a bit. When we create an Automation Rule, we need to give the rule a name. It should be descriptive and indicative of what the rule is going to do and what conditions it applies to. For example, a rule that automatically resolves an incident based on a known false positive condition on a server named SRV02021 could be titled “Automatically Close Incident When Affected Machine is SRV02021” but really it’s up to you to decide what you want to name your rules. Trigger The next thing we need to define for our Automation Rule is the Trigger. Triggers are what cause the automation rule to begin running. They can fire when an incident is created or updated, or when an alert is created. Of the two options (incident based or alert based), it’s preferred to use incident triggers as they’re potentially the aggregation of multiple alerts and the odds are that you’re going to want to take the same automation steps for all of the alerts since they’re all related. It’s better to reserve alert-based triggers for scenarios where an analytic rule is firing an alert, but is set to not create an incident. Conditions Conditions are, well, the conditions to which this rule applies. There are two conditions that are always present: The Incident provider and the Analytic rule name. You can choose multiple criterion and steps. For example, you could have it apply to all incident providers and all rules (as shown in the picture above) or only a specific provider and all rules, or not apply to a particular provider, etc. etc. You can also add additional Conditions that will either include or exclude the rule from running. When you create a new condition, you can build it out by multiple properties ranging from information about the Incident all the way to information about the Entities that are tagged in the incident Remember our earlier Automation Rule title where we said this was a false positive about a server name SRV02021? This is where we make the rule match that title by setting the Condition to only fire this automation if the Entity has a host name of “SRV2021” By combining AND and OR group clauses with the built in conditional filters, you can make the rule as specific as you need it to be. You might be thinking to yourself that it seems like while there is a lot of power in creating these conditions, it might be a bit onerous to create them for each condition. Recall earlier where I said the process for the three ways of creating Automation Rules was generally the same except using the Incident Action route? Well, that route will pre-fill variables for that selected instance. For example, for the image below, the rule automatically took the rule name, the rules it applies to as well as the entities that were mapped in the incident. You can add, remove, or modify any of the variables that the process auto-maps. NOTE: In the new Unified Security Operations Platform (Defender XDR + Sentinel) that has some new best practice guidance: If you've created an automation using "Title" use "Analytic rule name" instead. The Title value could change with Defender's Correlation engine. The option for "incident provider" has been removed and replaced by "Alert product names" to filter based on the alert provider. Actions Now that we’ve tuned our Automation Rule to only fire for the situations we want, we can now set up what actions we want the rule to execute. Clicking the “Actions” drop down list will show you the options you can choose When you select an option, the user interface will change to map to your selected option. For example, if I choose to change the status of the Incident, the UX will update to show me a drop down menu with options about which status I would like to set. If I choose other options (Run playbook, change severity, assign owner, add tags, add task) the UX will change to reflect my option. You can assign multiple actions within one Automation Rule by clicking the “Add action” button and selecting the next action you want the system to take. For example, you might want to assign an Incident to a particular user or group, change its severity to “High” and then set the status to Active. Notably, when you create an Automation rule from an Incident, Sentinel automatically sets a default action to Change Status. It sets the automation up to set the Status to “Closed” and a “Benign Positive – Suspicious by expected”. This default action can be deleted and you can then set up your own action. In a future episode of this blog we’re going to be talking about Playbooks in detail, but for now just know that this is the place where you can assign a Playbook to your Automation Rules. There is one other option in the Actions menu that I wanted to specifically talk about in this blog post though: Incident Tasks Incident Tasks Like most cybersecurity teams, you probably have a run book of the different tasks or steps that your analysts and responders should take for different situations. By using Incident Tasks, you can now embed those runbook steps directly in the Incident. Incident tasks can be as lightweight or as detailed as you need them to be and can include rich formatting, links to external content, images, etc. When an incident with Tasks is generated, the SOC team will see these tasks attached as part of the Incident and can then take the defined actions and check off that they’ve been completed. Rule Lifetime and Order There is one last section of Automation rules that we need to define before we can start automating the mundane away: when should the rule expire and in what order should the rule run compared to other rules. When you create a rule in the standalone automation UX, the default is for the rule to expire at an indefinite date and time in the future, e.g. forever. You can change the expiration date and time to any date and time in the future. If you are creating the automation rule from an Incident, Sentinel will automatically assume that this rule should have an expiration date and time and sets it automatically to 24 hours in the future. Just as with the default action when created from an incident, you can change the date and time of expiration to any datetime in the future, or set it to “Indefinite” by deleting the date. Conclusion In this blog post, we talked about Automation Rules in Sentinel and how you can use them to automate mundane tasks in Sentinel as well as leverage them to help your SOC analysts be more effective and consistent in their day-to-day with capabilities like Incident Tasks. Stay tuned for more updates and tips on automating Microsoft Sentinel!
