Top 5 Microsoft Sentinel Queries for Threat Hunting
Threat hunting in Microsoft Sentinel goes beyond relying on scheduled analytics rules. It’s about proactively asking better questions of your data to uncover stealthy or emerging attacker behavior before it turns into an incident. Effective hunting helps security teams spot activity that may never trigger an alert but still represents meaningful risk. Over time, these proactive hunts strengthen overall detection coverage and improve SOC maturity. In this post, I’ll walk through five high‑value Sentinel hunting queries that security teams can use to uncover suspicious activity across identity, endpoints, and cloud resources. Each example focuses on why the hunt matters and what attacker behavior it can reveal. To make these hunts actionable and measurable, each query is explicitly mapped to MITRE ATT&CK tactics and techniques. This alignment helps teams communicate coverage, prioritize investigations, and evolve successful hunts into repeatable detections. 1. Rare Sign‑In Locations for Privileged Accounts Why it matters Privileged identities are prime targets. A successful sign‑in from an unusual geography may indicate compromised credentials or token theft. What to hunt Look for successful sign‑ins by privileged users from locations they rarely use. // MITRE ATT&CK: T1078 (Valid Accounts), T1078.004 (Cloud Accounts) | Tactic: Initial Access SigninLogs | where ResultType == 0 | where UserPrincipalName has_any ("admin", "svc") | summarize count() by UserPrincipalName, Location | join kind=leftanti ( SigninLogs | where TimeGenerated < ago(30d) | summarize count() by UserPrincipalName, Location ) on UserPrincipalName, Location What to investigate next Conditional Access policies applied MFA enforcement status Correlation with device compliance or impossible travel alerts 2. Multiple Failed Logons Followed by Success Why it matters This pattern often indicates password spraying, brute force activity, or attackers testing credential validity before gaining access. What to hunt // MITRE ATT&CK: T1110 (Brute Force), T1110.003 (Password Spraying), T1110.001 (Password Guessing) | Tactic: Credential Access // Related: T1078 (Valid Accounts) once authentication succeeds SigninLogs | summarize Failed=countif(ResultType != 0), Success=countif(ResultType == 0) by UserPrincipalName, bin(TimeGenerated, 1h) | where Failed > 5 and Success > 0 What to investigate next IP reputation and ASN Whether failures span multiple users (spray behavior) Subsequent mailbox, SharePoint, or Azure activity 3. Unusual Process Execution on Endpoints Why it matters Attackers often use “living off the land” binaries (LOLBins) such as powershell.exe, wmic.exe, or rundll32.exe to evade detection. What to hunt // MITRE ATT&CK: T1059 (Command and Scripting Interpreter), // T1059.001 (PowerShell), T1059.003 (Windows Command Shell) | Tactic: Execution // Related: T1218 (Signed Binary Proxy Execution) when rundll32 and other signed binaries are abused DeviceProcessEvents | where FileName in~ ("powershell.exe", "wmic.exe", "rundll32.exe") | where InitiatingProcessFileName !in~ ("explorer.exe", "services.exe") | project TimeGenerated, DeviceName, FileName, ProcessCommandLine, InitiatingProcessFileName, InitiatingProcessCommandLine What to investigate next Encoded or obfuscated command lines Parent process legitimacy User context and device risk score 4. Newly Created or Modified Service Principals Why it matters Service principals are often abused for persistence or privilege escalation in Azure environments. What to hunt // MITRE ATT&CK: T1098 (Account Manipulation), T1098.001 (Additional Cloud Credentials) | Tactic: Persistence AuditLogs | where OperationName in ("Add service principal", "Update service principal") | project TimeGenerated, InitiatedBy, TargetResources, OperationName What to investigate next Assigned API permissions or directory roles Token usage after creation Correlation with unfamiliar IP addresses 5. Rare Azure Resource Access Patterns Why it matters Attackers exploring your environment often access subscriptions or resource groups they’ve never touched before. What to hunt // MITRE ATT&CK: T1526 (Cloud Service Discovery), T1069.003 (Permission Groups Discovery: Cloud) | Tactic: Discovery AzureActivity | summarize count() by Caller, ResourceGroup | join kind=leftanti ( AzureActivity | where TimeGenerated < ago(30d) | summarize count() by Caller, ResourceGroup ) on Caller, ResourceGroup What to investigate next Role assignments for the caller Whether access aligns with job function Any subsequent configuration changes Summary Table This table summarizes each Sentinel threat hunting query and maps it directly to the corresponding MITRE ATT&CK tactic and technique. By aligning hunts to ATT&CK, security teams can clearly communicate what adversary behaviors are being proactively investigated and identify gaps in coverage. This mapping also makes it easier to prioritize hunts, measure maturity, and transition high‑value hunts into analytics rules over time. Sentinel Hunt MITRE Tactic MITRE Technique Rare privileged sign‑ins Initial Access T1078 – Valid Accounts Failed then successful logons Credential Access T1110 – Brute Force LOLBin execution Execution T1059 / T1218 Service principal changes Persistence T1098.001 Rare resource access Discovery T1526 / T1069.003 Final Thoughts Threat hunting in Microsoft Sentinel is most effective when it’s continuous, hypothesis‑driven, and contextual. These queries are starting points, not finished detections. Tune them based on your environment, enrich them with UEBA insights, and align your hunts to MITRE ATT&CK techniques, as outlined in your existing Sentinel content strategy. If you consistently run hunts like these, you’ll catch suspicious behavior before it triggers an alert or before an attacker reaches their objective.Unlocking value with Microsoft Sentinel data lake
As security telemetry explodes and AI‑driven defense becomes the norm, it is critical to centralize and retain massive volumes of data for deep analysis and long‑term insights. Security teams are fundamentally rethinking how they manage, analyze, and act on security data. The Microsoft Sentinel data lake is a game changer for modern security operations, providing the foundation for agentic defense, deeper insights, and graph‑based enrichment. Security teams can centralize signals, simplify data management, and run advanced analytics, without compromising costs or performance. Across industries, organizations are using the Sentinel data lake to unify distributed data, search across years of telemetry, correlate sophisticated threats using graph-powered analytics, and operationalize agentic workflows at scale, turning raw security data into actionable intelligence. In this blog we will highlight some of the ways Sentinel data lake is transforming modern security operations. Unified, cost-effective security data foundation The challenge Many organizations tell us they have been forced to make difficult tradeoffs: high ingestion costs meant selectively choosing which logs to keep, often leaving data that might have been critical during an investigation. This selective logging creates blind spots, fragmented visibility, and unnecessary operational complexity across security operations. As a result, CISOs increasingly view selective logging as a material security risk to their organizations. How Sentinel data lake helps The Sentinel data lake removes these constraints by providing a cost‑effective, security‑optimized foundation for centralizing large volumes of security data. With the data lake, security teams can finally retain the breadth of telemetry they need without the financial penalties traditionally associated with long‑term security data retention. Organizations benefit from: A unified security data foundation designed to simplify investigations Long‑term, cost‑effective retention for up to 12 years Flexible querying across high‑volume data sets 6x data compression in storage, enabling significantly lower retention costs at scale Why it matters By unifying data in a purpose-built security data lake, SOC teams gain reliable, comprehensive visibility without the budget limitations that once forced them to choose between cost and completeness. This stronger foundation not only improves day‑to‑day investigations; it unlocks the advanced analytics and AI‑powered capabilities that future proof SOCs for AI driven defense. With full visibility restored, organizations are better equipped to identify emerging threats, respond with confidence, and modernize their security operations on their own terms. Historical security analysis The challenge SOC teams often struggle with short SIEM retention windows that limit how far back investigators can look. Critical logs age out before teams can fully piece together an attack, making root‑cause analysis slow and incomplete. This challenge grows when incidents span long periods, when new threat indicators emerge, or when organizations need to understand how a compromise evolved over time. Without access to historical telemetry, analysts face significant blind spots that weaken both investigations and hunting efforts. How Sentinel data lake helps The Sentinel data lake solves this by enabling organizations to retain and analyze years of security data at a fraction of the cost of traditional SIEM retention. Teams can use KQL and notebooks to run deep, long‑range investigations, perform advanced anomaly detection, and correlate older events that would have been impossible to recover in the analytics tier. Historical data enables retro analysis when new threat intel emerges. SOC teams can instantly look back to validate whether newly discovered indicators, techniques, or threat actors were already present in their environment. Organizations benefit from: Years of cost‑effective retention that extend far beyond traditional SIEM windows Deep forensic investigations using KQL and notebooks over historical data Improved anomaly detection with long‑range patterns and baselines Faster scoping of incidents with access to full historical context Why it matters By unlocking access to years of searchable telemetry, SOC teams are no longer limited by short retention windows or forced to make compromises that weaken security. They can retrace the full scope of an incident, hunt for slow‑moving threats, and quickly respond to new IOCs, powered by the historical context modern attacks demand. This long‑range visibility strengthens both detection and response, giving organizations the confidence and continuity they need to stay ahead of evolving threats. Graph-powered attack-path visibility and entity correlation The challenge Traditional investigations often rely on reviewing logs in isolation, making it difficult to connect identity activity, endpoint behavior, cloud access, and threat intelligence in a meaningful way. As a result, SOC teams find it difficult to trace attack paths, understand lateral movement, and build complete investigative context. Without a unified view of how entities relate to each other, investigations become slow, fragmented, and are prone to missed signals. How Sentinel data lake helps The Sentinel data lake enables powerful graph‑based correlation across identity, asset, activity, and threat intelligence data. Using graph models, analysts can visually explore how entities connect, identify hidden attack paths, pinpoint exposed routes to sensitive assets, and understand the full blast radius of compromised accounts or devices. This graph‑driven context turns complex telemetry into intuitive visuals that dramatically accelerate both pre‑breach context and incident response. Organizations benefit from: Graph‑powered correlation across identity, asset, activity, and threat intelligence data Visualization of attack paths and lateral movement that logs alone cannot expose Context‑rich investigations supported by relationship‑driven insights Greater cross‑domain visibility that strengthens both detection and response Why it matters With graph‑powered context, SOC teams move beyond event‑by‑event analysis and gain a deep understanding of how their environment behaves as a system. This visibility speeds investigations, strengthens posture before attackers strike, and provides analysts with a clear, intuitive way to uncover relationships that traditional log searches simply can’t reveal. Agentic workflows powered by MCP server The challenge SOC teams are under constant pressure from rising alert volumes, repetitive manual investigative steps, and skill gaps that make consistent triage challenging. Even experienced analysts struggle to reason across large, distributed datasets, and junior analysts often lack the experience needed to understand complex threat scenarios. These challenges slow down response and increase the risk of missed signals. How the Sentinel data lake helps The Sentinel data lake, combined with the Model Context Protocol (MCP), enables AI agents to reason over unified, contextual security data using natural‑language prompts. Analysts can ask questions directly: “Does this user have other suspicious activity?” or “What assets are at risk?”, and agents automatically interpret the request, query the data lake, and return actionable insights. These AI‑powered workflows reduce repetitive effort, strengthen investigative consistency, and help teams operate at a higher level of speed and precision. Organizations benefit from: AI‑assisted investigations that reduce manual effort and accelerate triage Agentic workflows powered by MCP to automate multi‑step reasoning over unified data Natural‑language interactions that make complex queries accessible to all analysts Consistent, high‑quality analysis regardless of analyst experience level Why it matters By introducing agentic, AI‑driven workflows, SOC teams can automate time‑consuming tasks, reduce alert fatigue, and empower every analyst, regardless of seniority, to quickly arrive at high‑quality insights. This shift not only accelerates investigations but also frees teams to focus on high‑value, proactive security work. As organizations continue modernizing their SOC, agentic workflows represent a major step forward in bridging the gap between human expertise and scalable, AI‑powered analysis. The future of security operations starts here The Sentinel data lake is becoming the backbone of modern security operations—unifying security data, expanding investigative reach, and enabling graph‑driven, AI‑powered analysis at scale. By centralizing telemetry on a cost‑effective, AI‑ready foundation, and running advanced analytics on that data, security teams can move beyond fragmented insights to correlate threats with clarity and act faster with confidence. These four use cases are just the beginning. Whether you’re strengthening investigations, advancing threat hunting, operationalizing AI, or preparing your SOC for what’s next, the Sentinel data lake provides the scale, intelligence, and flexibility to reduce complexity and stay ahead of evolving threats. Now is the time to accelerate toward a more resilient, adaptive, and future‑ready security posture. Get started with Microsoft Sentinel data lake todayIntroducing the next generation of SOC automation: Sentinel playbook generator
Security teams today operate under constant pressure. They are expected to respond faster, automate more, and do so without sacrificing precision. Traditional security orchestration, automation and response (SOAR) approaches have helped, but they still rely heavily on rigid templates, limited action libraries, and workflows stretched across multiple portals. Building and maintaining automation is often slow and constrained at exactly the time organizations need more flexibility. Something needs to change – and with the introduction of AI and coding models the future of automation is going to look very different than it is today. Today, we’re introducing the Microsoft Sentinel playbook generator, a new way to design code-based playbooks using natural language. With the introduction of generative AI and coding models, coding itself is becoming democratized, and we are excited to bring these new capabilities into our experience. This release represents the first milestone in our next‑generation security automation journey. The playbook generator allows users to design and generate fully functional playbooks simply by describing what they need. The tool generates a Python playbook with documentation and a visual flowchart, streamlining workflows from creation to execution for greater efficiency. This approach is highly flexible, allowing users to automate tasks like team notifications, ticket updates, data enrichment, or incident response across Microsoft and third-party tools. By defining an Integration Profile (base URL, authentication, credentials), the playbook generator can create API calls dynamically without needing predefined connectors. The system also identifies missing integrations and guides users to add them from the Automation tab or within the authoring page Users especially value this capability, allowing for more advanced automations. Playbook creation starts by outlining the workflow. The playbook generator asks questions, proposes a plan, then generates code and documentation once approved. Users can validate playbooks with real alerts and refine code anytime through chat instructions or manual edits. This approach combines the speed of natural language with transparent code, enabling engineers to automate efficiently without sacrificing control or flexibility. Preview customers report that the playbook generator speeds up automation development, simplifies automations for teams, and enables flexible workflow customization without reliance on templates. The playbook generator focuses on fast, intuitive, natural‑language‑driven automation creation, supported by a powerful coding foundation. It aligns with how security teams want to work: flexible, integrated, and deeply customizable. We’re excited to see how customers will use this capability to simplify operations, eliminate repetitive work, and automate tasks that previously demanded deep engineering effort. This marks the start of a new chapter, as AI continues to evolve and reshape what’s possible in security automation. How to get started With just a few prerequisites in place, you can begin creating code‑based automations through natural‑language conversations, directly inside the Microsoft Defender portal. Here’s a quick guide to help you move from first steps to your first generated playbook: 1. Make sure the prerequisites are in place Before you open your first chat in the playbook generator, the AI coding agent behind the playbook generator, confirm that your environment is ready: Security Copilot enabled: Your tenant must have a Security Copilot workspace, configured to use a Europe or US-based capacity. Sentinel workspace in the Defender portal: Ensure your Microsoft Sentinel workspace is onboarded to the Microsoft Defender portal. 2. Ensure you have the right permissions To build and deploy generated playbooks, make sure you have the same permissions required to author Automation Rules—the Microsoft Sentinel Contributor role on the relevant workspaces or resource groups. 3. Configure your integration profiles Integration profiles allow the playbook generator to create and execute any dynamic API calls—one of the most powerful capabilities of this new system. Before you create your first playbook: Go to Automation → Integration Profiles in the Defender portal. Create a Graph API Integration Create Integration to the services you want to have in the playbook (Microsoft Graph, ticketing tools, communication systems, third‑party providers, or others). Provide the base URL, authentication method, and required credentials. 4. Create your first generated playbook From the Automation tab: Select Create → Generated Playbook. Give your playbook a name. 3. The embedded Visual Studio Code window opens— Start in plan mode by simply describing what you want your automation to do. Be explicit about: What data to extract What actions to perform Any conditions or branches Example prompt you can use: “Based on the alert, extract the user principal name, check if the account exists in Entra ID, and if it does, disable the account, create a ticket in ServiceNow, and post a message to the security team channel.” The playbook generator will guide the process, ask clarifying questions, propose a plan, and then—once approved—switch to Act mode to generate the full Python playbook, documentation with a visual flow diagram, and tests. Completing your first playbook marks the beginning of a more intuitive, responsive, and intelligent automation experience—one where your expertise and AI work side by side to transform how your SOC operates. This is more than a new tool; it’s a foundation that will continue to evolve, adapt, and empower defenders as security automation enters its next era. Watch a demo here: https://aka.ms/NLSOARDEMO For deeper guidance, advanced scenarios, and end‑to‑end instructions, you can explore the full playbook generator documentation: Generate playbooks using AI in Microsoft Sentinel | Microsoft Learn.Accelerate Your UEBA Journey: Introducing the Microsoft Sentinel Behaviors Workbook
In our recent announcement, we introduced the UEBA Behaviors layer - a breakthrough capability that transforms noisy, high-volume security telemetry into clear, human-readable behavioral insights. The Behaviors layer answers the critical question: "Who did what to whom, and why does it matter?" by aggregating and sequencing raw events into normalized, MITRE ATT&CK-mapped behaviors. But understanding what behaviors are is just the beginning. The next question for SOC teams is: "How do I actually use behaviors to get value from day one?" Today, we announce the Microsoft Sentinel Behaviors Workbook (part of the “UEBA essentials solution” in the content hub) - a purpose-built, interactive analytics workbook that helps you extract maximum value from the Behaviors layer across your investigation, hunting, and detection workflows. Whether you're a SOC manager looking for high-level situational awareness, an analyst triaging an incident, or a threat hunter searching for hidden threats, this workbook provides the insights you need, when you need them. And the best thing? You can always make it your own! Why a Workbook? While the behaviors data is incredibly rich, knowing where to start and what questions to ask can present a learning curve. The UEBA Behaviors Workbook solves this by providing pre-built, validated analytics across three core SOC workflows: Overview: High-level metrics and trends for leadership and SOC managers Investigation: Deep-dive timeline analysis for incident response Hunting: Proactive threat discovery with anomaly detection and attack chain analysis Think of the workbook as your guided tour through the Behaviors layer - it surfaces the most actionable insights automatically, while still giving you the flexibility to drill down and customize as needed. Quick Recap: What Are UEBA Behaviors? Before diving into the workbook, let's briefly recap what makes the Behaviors layer unique: Behaviors are neutral, descriptive observations - they explain what happened, not whether it's malicious. They aggregate and sequence raw events from sources like AWSCloudTrail, GCPAuditLog and CommonSecurityLog data into unified, human-readable summaries. Each behavior is enriched with MITRE ATT&CK mappings, entity roles, and natural-language explanations. They bridge the gap between raw logs and alerts, providing an abstraction layer that makes investigation and detection dramatically faster. In essence: behaviors turn "what's in the logs" into "what actually happened in my environment" - without requiring deep expertise in every log schema. The Behaviors Workbook: Three Tabs, Three Workflows The Behaviors Workbook is organized into three tabs; each designed for a specific SOC persona and use case. Let's walk through each one. Tab 1: Overview - Situational Awareness at a Glance Who it's for: SOC managers, leadership, and anyone who needs a quick pulse-check on what's happening in the environment. What it provides: The Overview tab delivers high-level metrics and visualizations including key metrics tiles, timeline trend charts, MITRE coverage heatmaps, and behavior type distribution that help you quickly spot spikes, patterns, or anomalies requiring investigation. Use case example: A SOC manager opens the Overview tab and immediately sees an unusual spike in behaviors concentrated in the Defense Evasion and Persistence tactics. The Behavior Type Distribution reveals a surge in "Failed IAM Identity Provider Configuration Attempts" and "AWS EC2 Security Group Rule Modifications Observed", signaling potential attack preparation that needs immediate triage. Tab 2: Hunting - Proactive Threat Discovery Who it's for: Threat hunters, purple teams, and proactive security analysts. What it does: The Hunting tab empowers hunters to discover emerging threats before they become incidents by surfacing anomalous patterns, rare behaviors, and potential attack chains. Unlike the Investigation tab (which reacts to known incidents), Hunting is about proactive discovery. Key capabilities: Use case example: Rarest Behaviors A threat hunter reviews the "Rarest Behaviors" panel, filtered for the past 7 days. They notice a behavior titled "Inbound remote management session from external address" that has only occurred 5 times in the entire environment. Pivoting to the BehaviorEntities table, they discover all 5 instances involve Palo Alto firewall logs showing the same external IP targeting different internal management interfaces - a clear sign of targeted reconnaissance. Use case example: Attack Chain Detector The Attack Chain Detector highlights an AWS IAM role (arn:aws:iam::123456789012:role/CompromisedRole) appearing across 5 distinct MITRE tactics: Reconnaissance, Persistence, Defense Evasion, Credential Access, and Impact. Reviewing the associated behaviors reveals: This multi-stage pattern - invisible when looking at individual CloudTrail events - is now crystal clear. The hunter initiates an immediate investigation. Use case example: CyberArk Vault Anomaly The workbook shows that the "CyberArk Vault CPM Automatic Detection Operations" behavior had an average of 120 instances per day over the past week, but today it has 1,847 instances - a 15x increase. Drilling into the behaviors reveals that a single service account is performing mass privileged account access operations across multiple safes - potential insider threat or compromised privileged account. This insightful information would have been buried in verbose Vault audit logs, but velocity tracking surfaces it immediately. Tab 3: Investigation - Deep-Dive Analysis for Incident Response Who it's for: SOC analysts, incident responders, and anyone investigating a specific incident or specific entities. What it does: The Investigation tab transforms how analysts respond to incidents by providing comprehensive behavioral context for the entities involved. Instead of manually querying multiple log sources and stitching together timelines, analysts get an automated, pre-correlated view of everything those entities did before, during, and after the incident. How to use it: When investigating an incident, you provide: The entities involved (users, machines, IPs, etc.) The time of incident generation Time range before the first alert (e.g., 24 hours before) Time range after the last alert (e.g., 12 hours after) Use case example: An alert fires for "Suspicious AWS IAM Activity" involving IAM user AdminUser123. The analyst opens the Investigation tab, enters the user identity as the entity, sets the incident time, and configures a 24-hour lookback and 12-hour look-forward window. The analyst immediately sees: Before the incident: Normal behaviors like "AWS EC2 Security Group Information Retrieval" show routine reconnaissance. During the incident: Multiple instances of "Failed IAM Identity Provider Configuration Attempts", indicating the attacker is trying to establish persistence through SAML federation After the incident: "AWS Resource Deletion Monitoring" behaviors showing potential attempted cleanup of evidence. This comprehensive view - which would have taken 30+ minutes of manual querying across CloudTrail, VPC Flow Logs, and IAM logs - is now available in seconds and is easily readable and provides rich context. Real Impact on Your SOC The Behaviors Workbook represents a fundamental shift in how SOCs can operate: Investigation time drops from hours to minutes through automated entity-centric behavioral analysis. Threat hunting becomes accessible to junior analysts through pre-built queries that surface rare behaviors and attack chains. Leadership gains visibility into MITRE ATT&CK coverage and behavior trends without needing to know KQL. Detection engineering is faster because rare behaviors and velocity anomalies are automatically surfaced as high-fidelity signals. The workbook doesn't just give you data - it gives you insights you can act on immediately. Getting Started Prerequisites: A Microsoft Sentinel workspace onboarded to the Microsoft Defender portal. The Behaviors layer enabled for your workspace (Settings → UEBA → Behaviors layer) and at least one supported data source configured (list is always updated in the documentation). The Workbook uses the Log Analytics table names, SentinelBehaviorInfo and SentinelBehaviorEntities. The “Sentinel” prefix isn’t needed when querying behaviors in Advanced Hunting. Installation: Navigate to Microsoft Sentinel → Content Hub. Search for the "UEBA essentials" solution in the gallery. Click Save to add it to your workspace. One of the content items is the UEBA behaviors workbook (you will also find there a Workbook for UEBA and great hunting queries to get you started with UEBA). Open the workbook and select your time range and parameters. Adjust the queries as needed for your use cases. We Want Your Feedback As you start using the workbook, let us know: Which tab do you find most valuable? What additional visualizations or hunting queries would help your workflow? What should be integrated into the portal, and where? Share your thoughts in the comments below or reach out to our team directly. For more details on the Behaviors layer, see our original announcement blog post and https://learn.microsoft.com/en-us/azure/sentinel/entity-behaviors-layer. You will find those links in the “Resources” tab of the Workbook for ease of use.Disrupt AWS attacks automatically in Microsoft Sentinel
Attacks move faster than security teams can react. They spread across identities, endpoints, and SaaS apps in minutes, overwhelming analysts with signals and leaving little time to act. By the time an incident is investigated, the attacker has often already moved on escalating impact and business continuity. Organizations need a way to respond and get ahead of these attacks. That’s why we built automatic attack disruption, Microsoft’s AI‑driven self‑defense that stops in‑progress, multi‑domain attacks like ransomware in minutes before they can cause damage. At Ignite, we expanded attack disruption to support critical data sources ingested through Microsoft Sentinel: Amazon Web Services (AWS) and Proofpoint. This enables real-time detection and automatic containment of threats like phishing and identity compromise on top of your log data, fundamentally turning your SIEM into a threat protection solution. Built on the industry’s most comprehensive XDR, Microsoft Defender applies Microsoft’s vast threat intelligence, deep security research, and powerful threat protection capabilities across any security signal so every log source in a customer’s environment benefits from the same high-fidelity detection, investigation, and response. AWS: From Initial Access to real‑time containment As organizations increasingly build and run critical workloads on AWS, the cloud has become one of the most attractive and frequently targeted attack surfaces for modern threat actors. With 45% of all data breaches in 2025 involving cloud‑based assets and 81% of organizations experiencing at least one cloud security incident in the past year, attackers are capitalizing on exposed identities at unprecedented scale. To safeguard customers, attack disruption now disrupts two identity scenarios that most often drive attacker progression. 1. Compromised AWS federated user In this scenario, a compromised Entra ID account is used to access AWS resources by assuming a federated AWS role. Automatic attack disruption will automatically disable the Entra ID user account and revoke the session, preventing the attacker from performing any further actions. Additionally, the federated AWS session will be revoked (via a deny policy) to immediately cut off the attacker’s activity in AWS. 2. Compromised AWS IAM user In this scenario, an AWS IAM account is compromised by the attacker. Attack disruption contains the account by applying a deny policy, which restricts any further activity from the compromised account in AWS. Let’s look at a real-world scenario where attack disruption stops an attack on AWS. In this incident, we can see the activity leading up to the attack in AWS and that it was automatically contained by attack disruption. Replaying the sequence: The first indication is a phishing campaign where emails were deleted after delivery. Following this, a suspicious sign‑in from the compromised user account appears, along with a new network connection, signaling potential account takeover. The attacker then uses the victim’s Entra ID credentials to federate into a privileged AWS account. With signals from Sentinel correlated with XDR, Defender reaches high‑confidence confirmation of compromise. Attack disruption automatically revokes the session token and disables both the compromised Entra ID account and the AWSAdminRole used by the attacker. But the attacker attempts to pivot by leveraging a secondary backdoor AWS account they had created earlier. Defender immediately detects this attempt and disables the backdoor account as well, preventing further lateral movement and neutralizing the intrusion completely. Coming back to the incident, an additional reconnaissance alert appears based on AI‑generated signal from the Security Copilot Dynamic Threat Detection Agent. This agent investigates incidents to reveal hidden or correlated attacker activity, uncovering more alerts, assets, and indicators. It enriches the attack story and accelerates response by providing a dynamically generated “What Happened” explanation that clarifies the suspicious behavior and why the alert was raised. Together, Defender’s AI-powered capabilities combined with Security Copilot agents demonstrate how modern SOC operations evolve from reactive triage to proactive, high‑impact defense. Summary By bringing your AWS data into Sentinel, you not only gain deep visibility and detection coverage, but you also unlock powerful AI-driven capabilities like automatic attack disruption through Microsoft Defender. These signals fuel protection, helping you stay ahead of attackers by accelerating response and reducing impact. Getting started Attack disruption uses telemetry ingested via the AWS S3 connector. See the documentation for setup requirements Read the Ignite 2025 news Discover and deploy content from the Content HubNew content types supported in multi-tenant content distribution
Onboard new tenants and maintain a consistent security baseline We’re excited to announce a set of new content types that are now supported by the multi-tenant content distribution capability in the Defender portal: You can now distribute analytics rules, automation rules, workbooks, and alert tuning built in rules. What is content distribution? Content distribution is a powerful multi-tenant feature that enables scalable management of security content across tenants. With this capability, you can create content distribution profiles in the multi-tenant portal that allow you to seamlessly replicate existing content—such as custom detection rules and endpoint security policies—from a source tenant to designated target tenants. Once distributed, the content runs on the target tenant, enabling centralized control with localized execution. This allows you to onboard new tenants quickly and maintain a consistent security baseline across tenants. New supported content types With this release, we add support for several new content types: Analytics rules (Sentinel) Automation rules (Sentinel) Workbooks (Sentinel) Alert tuning rules (built-in rules) Soon, we will introduce more content types, including URBAC roles. How it works Navigate to ‘Content Distribution’ in Defender’s multi-tenant management portal Create a new distribution profile or select an existing distribution profile In the ‘Content selection’ step, select one of the new content types to distribute After choosing the content types, select the actual content that you want to distribute. For example, select analytics rules that you want to distribute to other tenants Use the filters to select which tenant (and workspace) to take the content from Choose at least one workspace that you want to distribute the content to. You can select up to 100 workspaces per tenant. Save the distribution profile and the content will be synced to your target tenants Review the sync result in your distribution profile Good to know Automation rules that trigger a playbook cannot currently be distributed Alert tuning rules are currently limited to distributing built-in rules, this will be expanded to custom rules later Learn more For more information, see Content distribution in multitenant management. To get started, navigate to Content distribution. FAQ What pre-requisites are required? Access to more than one tenant, with delegated access via Azure B2B, using multi-tenant management A subscription to Microsoft 365 E5 or Office E5 What permissions are needed to distribute? Each content type requires you to have permission to create that content type on the target tenant. For example, to create Analytics Rules, you require ‘Sentinel Contributor’ permissions. To distribute content using multi-tenant management content distribution, the Security settings (manage) or Security Data Basic (read) permission is required. Both roles are assigned to the Security Administrator and Security Reader Microsoft Entra built-in roles by default. Can I update or expand distribution profiles later? Yes. You can add more content, include additional tenants, or modify scopes as needed.Three ways to run KQL on Microsoft Sentinel data lake: Interactive, Async, or Jobs
SOC analysts often face complex challenges during investigations. They often need to investigate incidents that span weeks or even months, not just hours or days. This requires correlating multiple high-volume data sources such as sign-ins, network traffic, endpoint activity, and logs. Analysts frequently need to revisit query results throughout an active investigation and must determine whether findings are temporary insights or should evolve into long-term detections. Hunting and anomaly detection, in particular, demand access to months of historical data to uncover patterns and validate threats. With Microsoft Sentinel data lake, you can run KQL directly on data in the lake, unlocking flexibility for investigations and hunting. The key is choosing the right execution model for your scenario. Quick, interactive queries are great for initial exploration but may time out on complex queries or large datasets. Deep investigations require a more robust approach, while operationalized security outcomes call for scheduled jobs. “How should I run my KQL queries on the data lake to balance between efficiency, and speed?” If you’ve ever run a long query, waited… waited some more… and then hit a timeout, you’ve already met the limits of interactive KQL queries. The good news is that Sentinel data lake gives you multiple ways to query large volumes of security data, each designed for a different purpose. In this post, we’ll walk through: When to use interactive KQL queries When to run to KQL Async queries When should you consider using KQL jobs We'll explain how these options work in real SOC workflows and guide you in selecting the right KQL execution mode, so you can avoid unnecessary frustration and re-runs. Understanding the three KQL options for Sentinel data lake When working with Sentinel data lake, you’ll typically choose between three query execution modes: Interactive KQL queries Async KQL queries KQL jobs They all use Kusto Query Language (KQL), but they differ in how long they run, how results are stored, and who they’re best for. Let’s break them down. 1. Interactive KQL queries: Ad-hoc, faster access and temporary results Analysts often begin with interactive queries because they are simple, fast, and ideal for exploring data. When to use interactive queries Use interactive queries when: You’re running queries for ad-hoc investigations You want to a validate a query before putting them in a KQL job for scheduling The dataset is small to moderate You only need to view results once You want immediate feedback Queries that are completed quickly (ideally within 2-3 minutes for an analyst to wait for the results interactively). Common use cases Checking recent network logs or sign-in failures (e.g. last 24 hrs.) Exploring a suspicious IP over a short time window (e.g. last 24 hrs.) Verifying a hypothesis during triage What to expect Designed for quick execution Best for short lookback periods Queries may time out between ~7–8 minutes Results do not persist beyond the session Interactive queries are ideal for exploratory analysis, but they may not be ideal for heavy lifting across large datasets or long lookbacks of data in lake. 2. Async queries: Long-running investigations reducing the risk of timeout An Async query is a new feature in data lake where things get interesting, especially for incident investigations involving larger datasets. Instead of waiting for results in real time, Async queries run in the background and a user can check progress in Async tab. Results are in data lake hot cache for quicker retrieval on demand, up to 24 hours after execution. When to use Async queries Async queries are a great fit when: You’re querying larger datasets, or you need to query a longer lookback window. KQL Async queries can run for up to one hour, and you suspect your query would time out interactively A small group of analysts needs fast access to the same results You don’t want to hydrate data into the Analytics tier nor create use a custom table for that You don't need to use the results in your detection rules Common use cases Exploring a suspicious IP over a time window Requiring data from lake for an incident investigation that multiple analysts need to access for a short period of time Key benefits Queries can run for up to one hour Results are stored in data lake hot cache Results remain available for up to 24 hours Multiple users can fetch results Fast re-access without re-running the query on cold storage No need to permanently move data into Analytics tier This makes Async queries especially useful during active incidents, where one or two SOC analysts may need to revisit results multiple times while pivoting their investigation. KQL Jobs: For persistent results or custom detection rules KQL jobs are designed for persistence and reuse. Instead of just querying data, a KQL job hydrates results from the data lake into the Analytics tier, either as a one-time job or on a schedule. When to use KQL jobs Use KQL jobs when: You need the results long-term Data should be available for detections or dashboards You want to schedule recurring queries Multiple teams or detections depend on the output Summarization of logs from data lake into Analytics tier Common use cases One-time: Incident investigations spanning larger datasets from lake Jobs: High volume log summarization, anomaly detection, IoC matching and similar use cases Use Microsoft Sentinel workbooks, building dashboards using data on top of analytics-tier Produce enriched datasets for ongoing monitoring Important considerations KQL jobs can run for up to one hour in the background Results are stored permanently (unless the custom table is deleted) Best when query output becomes part of your proactive hunting process Think of KQL jobs as turning raw lake data into a reusable security asset. Putting it all together: A sample investigation scenario Let’s walk through a realistic SOC scenario to see how these query types work together. Scenario: Suspicious IP activity over 90 days An analyst is investigating a potentially malicious IP address reported by threat intelligence. The IP may have been active over the past 90 days. Step 1: Start Interactive Let’s say you decided to store SecurityEvent logs in lake only mode. An analyst in your team begins with an interactive KQL query on data lake to quickly check recent activity: SecurityEvent | where TimeGenerated > ago(24h) | where IpAddress == "203.0.113.45" | summarize count() by EventLevelName This may run quickly and confirm suspicious activity, with a short lookback such as last 24 hours. Step 2: Switch to Async for scale To understand the full scope, the analyst expands the lookback to 90 days and joins multiple data sources for the same IoC. The query is may be too slow for an interactive execution. So they run it as an Async query: union SigninLogs, SecurityEvent, CommonSecurityLog | where TimeGenerated > ago(90d) | where IPAddress == "203.0.113.45" | summarize FirstSeen = min(TimeGenerated), LastSeen = max(TimeGenerated), EventCount = count() by IPAddress, Category After providing a name for query, the execution begins: The query runs in the background, and it may take few minute. You can always check the progress In Async queries tab: Once the query completes successfully, the results are cached in data lake. Over the next few hours, the analyst revisits the results multiple times while pivoting to related entities, without having to wait query execution on the cold storage. Step 3: Operationalize with a KQL Job The investigation reveals a recurring attack pattern that leadership wants monitored continuously. A KQL job is created to: Run once or in a schedule (by minutes, daily, weekly, monthly) Hydrate results into a custom Analytics table With results in Analytics tier, power custom detection rules and dashboard Now the insights move from investigation to ongoing defense. Read our previous blog posts on how to run KQL jobs on data lake. How to Choose the Right Option Here’s a simple way to decide: Need quick answers now? → Use Interactive queries Query is big, slow, or spans long timeframes? → Use Async queries Results must be reused, scheduled, or used in custom detection rules? → Use KQL jobs Each option exists to reduce friction at a specific stage of analysis, from curiosity, to investigation, to operationalization. Final thoughts Microsoft Sentinel data lake gives security teams flexibility at scale, when the right query mode is used at the right time. Interactive queries keep investigations fast and exploratory. Async queries unlock deep, long-running analysis with higher time out limits. KQL jobs turn insights into durable security capabilities. Still need to run a query on massive datasets or longer lookback? Try Notebooks capabilities on data lake. Once you choose the right option for your scenarios, querying data in the lake becomes less about limits, and more about possibilities. Happy hunting! Resources Get started with Microsoft Sentinel data lake and KQL today. Microsoft Sentinel data lake overview - Microsoft Security | Microsoft Learn KQL and the Microsoft Sentinel data lake - Microsoft Security | Microsoft Learn Create jobs in the Microsoft Sentinel data lake - Microsoft Security | Microsoft LearnHyperscale ML threat intelligence for early detection & disruption
Co-author: Amir Gharib In today's rapidly evolving cybersecurity landscape, the ability to swiftly identify and mitigate threats is more critical than ever. Attackers are increasingly well-resourced, enabling them to keep adding new components to their toolkits that keep their infrastructure fresh and hard to detect. Traditional labeling methods used to identify and block malicious infrastructure are struggling to keep up. At Microsoft, we recognize the pressing need for innovative solutions that not only keep pace with these threats but stay ahead of them. This past Ignite, we announced Threat Intelligence Tracking via Dynamic Networks (TITAN)—a groundbreaking approach that uses the power of machine learning to transform threat intelligence and attack disruption by automatically neutralizing malicious activity at scale. By leveraging real-time ML-driven analytics, TITAN uncovers previously hidden threat actor infrastructure, enabling the disruption capabilities built into our unified security operations platform to detect and stop attacks significantly earlier in the attack chain (Figure 1). The power of machine-scale threat intelligence TITAN represents a new wave of innovation built on Microsoft threat intelligence capabilities, introducing a real-time, adaptive threat intelligence (TI) graph that integrates first and third-party telemetry from the unified security operations platform, Microsoft Defender for Threat Intelligence, Microsoft Defender for Experts, and customer feedback. This graph employs guilt-by-association techniques to propagate known TI labels to unknown neighboring entities (e.g., IP, file, email) at machine scale. By analyzing relationships between entities, TITAN can identify attacker infrastructure before they are leveraged in attacks, providing an invaluable window of opportunity to prevent harm. Figure 1. Architectural overview of TITAN, comprising four key steps: (1) constructing a graph using telemetry from 1 st and 3 rd party detectors in the Unified Security Operations Platform, (2) integrating known threat intelligence from across Microsoft, (3) applying reputation propagation algorithms to classify previously unknown entities as either benign or malicious, and (4) updating the reputation score for each entity in the graph. By leveraging guilt-by-association methods, TITAN can swiftly identify hidden threat actor infrastructure through cross-organizational associations with known malicious entities within the TI graph. Specifically, we employ a semi-supervised label propagation technique that iteratively assigns reputation scores to nodes based on their neighbors’ scores, refining the graph’s score distribution until convergence. These high-confidence entity reputation scores empower the unified security operations platform to implement proactive containment and remediation actions via attack disruption. A key advantage of our constantly evolving threat intelligence is that we can provide clear and explainable reputation scores for each entity by examining the neighboring entities that contribute to the overall score. Preventing attacks before they happen Consider a scenario where TITAN detects unusual activity from a seemingly benign IP address that has connections to known malicious domains. Traditional systems might not flag this IP until after malicious activity is confirmed. However, TITAN's guilt-by-association techniques elevate the reputation score of the IP address, immediately triggering detection and disruption rules that block the threat before any damage occurs. With an impressive average macro-F1 score of 0.89 and a precision-recall AUC of 0.94, TITAN identifies millions of high-risk entities each week, enabling a 6x increase in non-file threat intelligence. Since its deployment, TITAN has reduced the time to disrupt by a factor of 1.9x while maintaining 99% precision, as confirmed by customer feedback and thorough manual evaluation by security experts—ultimately saving customers from costly security breaches. Dynamic threat intelligence graph construction At the heart of TITAN is a dynamic, time-evolving threat intelligence graph that captures complex relationships between millions of interlinked entities, alerts, and incidents. By combining telemetry across both 1 st and 3 rd party sources in the unified security operations platform, TITAN is uniquely positioned for comprehensive view of the threat landscape, essential for early detection and disruption. Key features include: Real-time updates – In cybersecurity, speed is critical. TITAN operates in real-time, with graph creation and reputation propagation algorithms running every hour. This frequency ensures that security teams receive fresh and active threat intelligence, enabling swift and effective responses to emerging threats. The ability to act quickly can mean the difference between thwarting an attack and being breached. Infusing security domain knowledge via edge weights – Edges in the TI graph carry weights that signify the strength or relevance of the relationships between entities. We introduce edge weight decay functions that automatically reduce edge weights based on the time elapsed since the edge was formed. This ensures that newer and more relevant relationships have a greater impact on reputation assessments, aligning the dynamic graph with the real-time nature of security incidents. Pruning outdated nodes and edges – To maintain the relevance and efficiency of the TI graph, we implement pruning mechanisms that remove nodes and edges when their weights fall below certain thresholds. This approach keeps the graph focused on the most current and meaningful connections, ensuring optimal performance. Evolving cybersecurity defense with TI TITAN represents a monumental step forward in the mission to protect organizations from cyber threats. By infusing the power of AI with advanced threat intelligence, we are equipping security teams with the tools they need to stay ahead of the attackers. This is only possible with a unified platform that consolidates threat intelligence across 1 st and 3 rd party workloads and products, organizations benefit not only from streamlining their security operations but also gain deeper insights into potential threats and vulnerabilities. TITAN is just one of the many examples of how powerful bringing together the full capabilities of an industry-leading cloud-native security information and event management (SIEM), comprehensive extended detection and response (XDR), and generative AI built specifically for cybersecurity. Integrating all of this data, advanced analysis, threat intel and automation enables an entirely new era of defense for security teams and we’re so energized by the potential. TITAN is just the start – look forward to new capabilities announced in the coming months. Learn More Check out our resources to learn more about our new approach to AI-driven threat intelligence, and our recent security announcements: See TITAN in action in the session delivered at Ignite Read the full paper on the TITAN architecture Read the Copilot for Security Guided Response paper & blog Read the unified security operations platform GA announcementUnlocking the power of Notebooks with Microsoft Sentinel data lake
Co-authors: Vandana Mahtani, Ashwin Patil Security operations are rapidly evolving, driven by AI and the need for scalable, cost-effective analytics. A key differentiator of the Sentinel data lake is its native integration with Jupyter Notebooks, which brings powerful data science and machine learning capabilities directly into security operations. Analysts can move beyond static queries to run interactive investigations, correlate large and diverse datasets, and prototype advanced analytics using familiar tools and languages. By combining notebooks with Sentinel’s security context, teams can build custom detection logic, enrich investigations with ML models, and automate complex workflows. The result is faster insights, deeper analysis, and more efficient security operations, enabling SOC teams to innovate and respond at the speed required by today’s threat landscape. Hunt with Sentinel notebooks Notebooks in Sentinel data lake give security teams a powerful, interactive way to investigate and hunt across their security data at scale: Query and analyze massive datasets: Run Spark queries across months or years of security telemetry (higher thresholds compared to KQL), uncovering slow-moving threats and persistent attack patterns. Automate threat hunting: Schedule recurring jobs to scan for matches against newly ingested indicators of compromise (IOCs), enabling continuous detection and investigation. Build and operationalize ML models: Use Python, Spark, and built-in libraries to create custom anomaly detection, alert enrichment, and predictive analytics workflows. Enrich alerts and investigations: Correlate alerts with firewall, Netflow, and other logs—often stored only in the data lake—to reduce false positives and accelerate triage. Collaborate and share insights: Notebooks provide a transparent, reproducible environment for sharing queries, visualizations through python libraries like plotly that is not natively available in Sentinel, and sharing findings across teams. Cost-Efficient, Scalable Analytics Sentinel data lake’s tiered storage and flexible retention means you can ingets and store all your raw telemetry—network logs, firewall data, and more—at a fraction of the cost of traditional solutions. Notebooks help you unlock the value of this data, transforming raw logs into actionable insights with minimal manual intervention. Notebooks and KQL Jobs in Microsoft Sentinel data lake Both notebooks and KQL jobs enable teams to query and analyze data within Microsoft Sentinel data lake, but they serve very different purposes. Dimension Notebooks (Spark Runtime) KQL Jobs (Data lake KQL Engine) Execution Model Distributed compute using Apache Spark; ideal for heavy ETL, transformation, or ML workloads; supports programmatic querying Query execution using the KQL engine optimized for analytical queries over structured datalake tier data. Language & Flexibility Full Python ecosystem (Pandas, PySpark, MLlib etc.) out of the box in the cluster; ideal for data wrangling, ML, and automation pipelines. Familiar KQL syntax — purpose-built for log analytics, filtering, and aggregation. Best for converting your expensive queries in the pipelines Data Access Direct access to raw and curated tables stored in data lake tiers. Can join across multiple workspaces or schemas. Access to data lake tier tables – which includes mirrored tables from Analytics tier as well as curated table from other jobs. Performance & Scale Highly scalable distributed compute and transformation-heavy workloads. Optimized for low-latency query response and cost-efficient read operations. Ideal for investigative queries. Use Case Fit Advanced analytics, feature engineering, baselining, anomaly detection, enrichment pipelines. Operational queries, scheduled detections, and validation jobs. Automation Can be orchestrated via scheduled Spark jobs within the VSCode extension. Supports end-to-end ETL + ML automation via Python and Spark notebooks. Can be scheduled and parameterized for recurring jobs (e.g., daily data quality checks or detection lookups). Collaboration & Reproducibility Shared notebooks with code, outputs, and markdown for team review and reproducibility. Shared job definitions and saved query templates can be maintained with version control; less narrative, more operational. Visualization Leverage advanced libraries (Plotly, Seaborn, Matplotlib) for rich visuals- all available in spark compute cluster. Jobs will output to tables and then can be used via KQL rendering (timechart, barchart) for validation or quick insights. Extensibility Currently limited libraries (Azure Synapse libraries 3.4) but support will be extended to Bring-your-own libraries and Python dependencies post-Fabric integration. Limited to native KQL functions; extensibility via job scheduling and data connections. Skill Profile Data scientist / advanced security analyst / data engineer. Detection engineer / SOC analyst / operational analytics. Cost model Advanced data insights meter based on vcore compute consumed Microsoft Sentinel Pricing | Microsoft Security Data lake query meter based on the GB processed Microsoft Sentinel Pricing | Microsoft Security In practice, modern SOC teams increasingly treat notebooks and KQL jobs as complementary, not competing. KQL for signal discovery → Notebook for pattern analysis Use Defender Out-of-the box rules or custom detections to surface an interesting low to medium fidelity signal (e.g., spike in hourly failed logons across tenants compared to baseline). Then pivot to the notebook for historical trend analysis across six months of data. Notebook for enrichment → KQL for operationalization A notebook creates a behavioral baseline table stored in the data lake. A KQL rule consumes that dataset daily to trigger alerts when deviations occur. Notebook pipelines → data lake → analytics tier → dashboard A scheduled notebook curates and filters raw logs into efficient partitioned data lake tables. These tables are then used via lake explorer for ad-hoc hunting campaigns. Once the workflow consistently provides good true positives, elevate to analytic tier and set up custom detection to operationalize it for near real time detection. Together, these workflows close the loop between research, detection, and response. Getting Started: From Query to Automation The enhanced notebook experience in Sentinel data lake makes it easy to get started: Author Queries with IntelliSense: Benefit from syntax and table name suggestions for faster, error-free query writing. Schedule Notebooks as Jobs: Automate recurring analytics, such as hourly threat intelligence matching or daily alert summarization. Monitor Job Health: Use the Jobs dashboard to track job status, completions, failures, and historical trends. Leverage GitHub Copilot: Get intelligent assistance for code authoring and troubleshooting. Use GitHub copilot plan mode to come up with boiler plate code for complex workflow notebooks and collaborate with Copilot to refine further to you use case. For a step-by-step guide, see the public documentation Exploring and interacting with lake data using Jupyter Notebooks - Microsoft Security | Microsoft Learn. Watch how to quickly start using Notebooks on Sentinel data lake - Getting Started with Apache Spark for Microsoft Sentinel data lake Real-world scenarios Here are some impactful ways customers are using notebooks in Sentinel data lake: Extended Threat Investigations: Query data older than 90 days to uncover slow-moving attacks like brute-force campaigns that span accounts and geographies. Behavioral Baselining: Build time-series models to establish normal behavior and identify unusual patterns, such as credential abuse or lateral movement. Retrospective Threat Hunting: React to emerging IOCs by running historical queries across the data lake, enabling rapid and informed response. ML-Powered Insights: Operationalize custom machine learning models for anomaly detection and predictive analytics, directly within the notebook environment. Real-World Example — Extending a Detection with a Notebook Challenge A security team wants to identify password spray activity occurring gradually over time — the same IPs attempting one or two logons per day across different users — without overwhelming the system with false positives or exceeding query limits. Using a notebook (Spark runtime), analysts extend investigation across months of raw SigninLogs stored in the data lake: Load 6–12 months of sign-in data directly from lake storage. Aggregate failed logons by IP, ASN, and user over time. Apply logic to find IPs with repeated low-frequency failures spread across many accounts. Build complex statistics-based scoring model to find IP ranges of potential slow password spray. Visualize long-term trends with Plotly or Matplotlib to expose consistent patterns. Write results back as a curated dataset for downstream detection via analytic tier with aggregated threat intel from your own organization. For complete walkthrough of Sentinel data lake Password Spray solution, check out our GitHub - Password Spray Detection – End-to-End Pipeline. The notebook uncovers subtle, recurring IP ranges performing slow password sprays - activity invisible to short-window rules. The enriched dataset with statistical risk scoring and categorization of high, medium and low can then be used by a custom detection to generate proactive alerts when those IPs reappear. Call to Action Ready to transform your security operations? Get started with Microsoft Sentinel data lake today. Explore the possibilities with notebooks in Sentinel data lake: Running notebooks on the Microsoft Sentinel data lake - Microsoft Security | Microsoft Learn Create and manage Jupyter notebook jobs - Microsoft Security | Microsoft Learn
