(Last updated July 2024)
In this blog post, we try to walk you through Microsoft Sentinel level 400 training and help you become a Microsoft Sentinel master.
Update July 2024
Microsoft Unified Security Operations Platform
Microsoft Sentinel is now generally available within the Microsoft unified security operations platform in the Microsoft Defender portal. The Microsoft unified security operations platform brings together the full capabilities of Microsoft Sentinel, Microsoft Defender XDR, and Microsoft Copilot in Microsoft Defender. For more information, see the following resources:
- Blog post: General availability of the Microsoft unified security operations platform
- Blog post: Unified Security Operations Platform - Technical FAQ!
- Microsoft Sentinel in the Microsoft Defender portal
- Connect Microsoft Sentinel to Microsoft Defender XDR
- Microsoft Copilot in Microsoft Defender
This XDR and Sentinel integration into the only unified security operations platform delivers full SIEM and XDR capabilities.
Already did the Sentinel Ninja Training? check what's new.
Curriculum
This training program includes 21 modules. The post includes a presentation for each module, preferably recorded (when still not, we are working on the recording) and supporting information: relevant product documentation, blog posts, and other resources.
The modules listed below are split into five groups following the life cycle of a SOC:
Part 1: Overview
- Module 0: Other learning and support options
- Module 1: Get started with Microsoft Sentinel
- Module 2: How is Microsoft Sentinel used?
Part 2: Architecting & Deploying
- Module 3: Workspace and tenant architecture
- Module 4: Data collection
- Module 5: Log Management
- Module 6: Enrichment: TI, Watchlists, and more
- Module W: Log transformation
- Module X: Migration
- Module Z: ASIM and Normalization
Part 3: Creating Content
- Module 7: The Kusto Query Language (KQL)
- Module 8: Analytics
- Module 9: SOAR
- Module 10: Workbooks, reporting, and visualization
- Module Y: Notebooks
- Module 11: Use cases and solutions
Part 4: Operating
- Module 12: A day in a SOC analyst's life, incident management, and investigation
- Module 13: Hunting
- Module 14: User and Entity Behavior Analytics (UEBA)
- Module 15: Monitoring Microsoft Sentinel's health
Part 5: Advanced Topics
- Module 16: Extending and Integrating using Microsoft Sentinel APIs
- Module 17: Bring your own ML
Part 1: Overview
Module 0: Other learning and support options
The Ninja training is a level 400 training. If you don't want to go as deep or have a specific issue, other resources might be more suitable:
- While extensive, the Ninja training has to follow a script and cannot expand on every topic. The FAQ companion to the Ninja Training tries to closed this gap.
- Microsoft Sentinel's official learning path is best if you want step-by-step training to use Microsoft Sentinel's features.
- You can now certify with the new SC-200 certification (Microsoft Security Operations Analyst) which covers Microsoft Sentinel. The SC-200 is not a Ninja Training certification, but the exam is largely based on Ninja Training materials, making it a good learning path for the certification. You may also want to consider the SC-900 certification (Microsoft Security, Compliance, and Identity Fundamentals), for a broader, higher level view of the Microsoft Security suite.
- Premier customer? You might want the on-site (or remote these days) 4 day Microsoft Sentinel Fundamentals Workshop. Contact your Customer Success Account Manager to arrange.
- Already a Ninja? Just keep track of what's new, or join our Private Preview program for an even earlier glimpse.
- Have a good feature idea you want to share with us? Let us know on the Sentinel user voice page.
- Have a specific issue? Ask (or answer other) on the Microsoft Sentinel Tech Community. As a last resort, send an e-mail to MicrosoftSentinel@microsoft.com.
Think you're a true Sentinel Ninja? Take the knowledge check and find out. If you pass the
knowledge check with a score of over 80% you can request a certificate to prove your ninja
skills!
1. Take the knowledge check here.
2. If you score 80% or more in the knowledge check, request your participation certificate
here. If you achieved less than 80%, please review the questions that you got it wrong, study
more and take the assessment again.
Module 1: Get started with Microsoft Sentinel
Haven't looked at Sentinel recently? This webinar summarizes what's new in Sentinel
between Jan and Aug 2023.
Microsoft Sentinel is a scalable, cloud-native, security information event management (SIEM) and security orchestration automated response (SOAR) solution. Azure Sentinel delivers intelligent security analytics and threat intelligence across the enterprise, providing a single solution for alert detection, threat visibility, proactive hunting, and threat response (read more).
If you want to get an initial overview of Microsoft Sentinel's technical capabilities, the latest Ignite presentation is a good starting point. You might also find the Quick Start Guide to Microsoft Sentinel useful (requires registration). A more detailed overview, however somewhat dated, can be found in this webinar: YouTube
Lastly, want to try it yourself? The Microsoft Sentinel All-In-One Accelerator (blog, Youtube) presents an easy way to get you started. To learn how to start yourself, review the onboarding documentation, or watch Insight's Sentinel setup and configuration video.
Learn from users
Thousands of organizations and service providers are using Microsoft Sentinel. As usual with security products, most do not go public about that. Still, there are some.
- You can find public customer use cases here,
- Insight released a use case about an NBA team adapting Sentinel.
- Stuart Gregg, Security Operations Manager @ ASOS, posted a much more detailed blog post from Microsoft Sentinel's experience, focusing on hunting.
Learn from Analysts
- Microsoft Sentinel is a Leader placement in Forrester Wave.
- Microsoft is again named a Leader in the 2024 Gartner Magic Quadrant for Security Information and Ev...
Module 2: How is Microsoft Sentinel used?
Many users use Microsoft Sentinel as their primary SIEM. Most of the modules in this course cover this use case. In this module, we present a few additional ways to use Microsoft Sentinel.
As part of the Microsoft Security stack
Use Sentinel and Microsoft 365 Defender in tandem to protect your Microsoft workloads, including Windows, Azure, and Office:
- Read more about our comprehensive SIEM+XDR solution combining Microsoft Sentinel and Microsoft 365 Defender.
- Read The Azure Security compass to understand Microsoft's blueprint for your security operations.
- Read and watch how such a setup helps detect and respond to a WebShell attack: Blog Post, Video demo.
- Watch the webinar: Better Together | OT and IoT Attack Detection, Investigation and Response.
To monitor your multi-cloud workloads
The cloud is (still) new and often not monitored as extensively as on-prem workloads. Read this presentation to learn how Microsoft Sentinel can help you close the cloud monitoring gap across your clouds.
Side by side with your existing SIEM
Either for a transition period or a longer term, if you are using Microsoft Sentinel for your cloud workloads, you may be using Microsoft Sentinel alongside your existing SIEM. You might also be using both with a ticketing system such as Service Now.
For more information on migrating from another SIEM to Microsoft Sentinel, watch the migration webinar: MP4, YouTube, Deck.
The new Microsoft Sentinel Migration experience helps customers and partners automate the process of migrating their security monitoring use cases hosted in non-Microsoft products into Microsoft Sentinel.
- This first version of the tool supports migrations from Splunk
Watch the webinar on the Splunk to Microsoft Sentinel Migration Experience
There are three common scenarios for side by side deployment:
- A best practice, if you have a ticketing system in your SOC, is to send alerts, or incidents, from both SIEM systems to a ticketing system such as Service Now, for example, using Microsoft Sentinel Incident Bi-directional sync with ServiceNow or by sending alerts enriched with supporting events from Microsoft Sentinel to 3rd party SIEMs.
- At least initially, many users send alerts from Microsoft Sentinel to your on-prem SIEM. Read on how to do it in Sending alerts enriched with supporting events from Microsoft Sentinel to 3rd party SIEMs.
- Over time, as Microsoft Sentinel covers more workloads, it is typical to reverse that and send alerts from your on-prem SIEM to Microsoft Sentinel. To do that:
- With Splunk, read "Send data and notable events from Splunk to Microsoft Sentinel using the Microsoft Sentinel Splunk ...."
- With QRadar read Sending QRadar offenses to Microsoft Sentinel
- For ArcSight, use CEF Forwarding.
You can also send the alerts from Microsoft Sentinel to your 3rd party SIEM or ticketing system using the Graph Security API , which is simpler but would not enable sending additional data.
For MSSPs
Since it eliminates the setup cost and is location agnostics, Microsoft Sentinel is a popular choice for providing SIEM as a service. You can find a list of MISA (Microsoft Intelligent Security Association) member MSSPs using Microsoft Sentinel. Many other MSSPs, especially regional and smaller ones, use Microsoft Sentinel but are not MISA members.
To start your journey as an MSSP, you should read the Microsoft Sentinel Technical Playbooks for MSSPs. More information about MSSP support is included in the next module, cloud architecture and multi-tenant support.
Part 2: Architecting & Deploying
While the previous section offers options to start using Microsoft Sentinel in a matter of minutes, before you start a production deployment, you need to plan. This section walks you through the areas that you need to consider when architecting your solution, as well as provides guidelines on how to implement your design:
- Workspace and tenant architecture
- Data collection
- Log management
- Threat Intelligence acquisition
Module 3: Workspace and tenant architecture
Short on time? Watch the Nic DiCola's Ignite presentation (first 11 Minutes)
Get Deeper? Watch the Webinar: MP4, YouTube, Presentation
A Microsoft Sentinel instance is called a workspace. The workspace is the same as a Log Analytics workspace and supports any Log Analytics capability. You can think of Sentinel as a solution that adds SIEM features on top of a Log Analytics workspace.
Multiple workspaces are often necessary and can act together as a single Microsoft Sentinel system. A special use case is providing service using Microsoft Sentinel, for example, by an MSSP (Managed Security Service Provider) or by a Global SOC in a large organization.
To learn more about why use multiple workspaces and use them as one Microsoft Sentinel system, read Extend Microsoft Sentinel across workspaces and tenants or, if you prefer, the Webinar version: MP4, YouTube, Presentation.
There are a few specific areas that require your consideration when using multiple workspaces:
- An important driver for using multiple workspaces is data residency. Read more about Microsoft Sentinel data residency.
- To deploy Microsoft Sentinel and manage content efficiently across multiple workspaces; you would like to manage Sentinel as code using CI/CD technology. This is, in general, a recommended best practice for Microsoft Sentinel:
- Read deploying and Managing Microsoft Sentinel - Ninja style for a comprehensive CI/CD methodology.
- Or use a simpler solution to deploy and Managing Microsoft Sentinel as Code and extend this capability across workspaces and tenants using Azure Lighthouse.
- When managing multiple workspaces as an MSSP, you may want to protect the MSSP’s Intellectual Property in Microsoft Sentinel.
The Microsoft Sentinel Technical Playbook for MSSPs provides detailed guidelines for many of those topics, and is useful also for large organizations, not just to MSSPs.
Module 4: Data collection
Watch the Codeless Connector Platform: Create your data connector in Microsoft Sentinel
webinar. Youtube, Deck.
Watch the Manage Your Log Lifecycle with New Methods for Ingestion, Archival, Search,
and Restoration webinar. YouTube, Deck.
Short on time? Watch the Nic DiCola's Ignite presentation (Mid 11 Minutes)
Get Deeper? Watch the Webinar: YouTube, MP4, Deck.
The foundation of a SIEM is collecting telemetry: events, alerts, and contextual enrichment information such as Threat Intelligence, vulnerability data, and asset information. You can find a list of sources you can connect here:
- Documentation of the connectors which are part of the connectors gallery.
- Microsoft Sentinel Solutions gallery to deliver instant out-of-the-box solutions and content using Content Hub.
- Data Collection Scenarios: Learn more about a variety of solutions for log collection methods such as Logstash/CEF/WEF and scenarios we often encounter such as permissions restriction to tables, log filtering, collecting logs from AWS/GCP, O365 raw logs and more in this webinar YouTube, MP4, Deck.
How you connect each source falls into several categories or source types. Each source type has a distinct setup effort but once deployed, it serves all sources of that type. The Grand List specifies for each source what its type is. To learn more about those categories, watch the Webinar (includes Module 3): YouTube, MP4, Deck.
The types are:
- Built-in service-to-service connectors allow Microsoft Sentinel to connect directly to cloud services such as Office 365 or AWS CloudTrail. Some of the service-to-service connectors, such as AAD, utilize Azure diagnostics behind the scenes.
- Direct refers to sources that natively know how to send data to Microsoft Sentinel or Log Analytics. These include Azure services or other Microsoft solutions that support sending telemetry (often referred to as "diagnostics") to Log Analytics and 3rd party sources that use the ingestion API to write to Log analytics or Microsoft Sentinel directly. The Microsoft direct sources are listed in addition to the Grand List and in the blog post "Collecting logs from Microsoft Services and Applications."
- The Log Forwarder is a VM that enables collecting Syslog and CEF events from remote systems. If a source is listed in the Grand List as CEF or Syslog, you will use the Log Forwarder to collect from it. Learn more about the Log Forwarder in this webinar (plus a bonus: learn how to use it to filter events): YouTube, MP4, Deck.
- The Azure Monitor Agent (AMA) is the replacement for the Log Analytics Agent. The Azure Monitor agent introduces several new capabilities not available in the Log Analytics agent such as filtering, scoping, and multi-homing. At the time of writing this update, AMA isn't yet at parity with the Log Analytics agent, you can compare the functionality between the two agents here. Consider whether the features you need for your Microsoft Sentinel deployment are supported in AMA, or whether to continue to use the Log Analytics agent for now and migrate at a later date. Watch the Everything You Ever Wanted to Know About Using the New Azure Monitor Agent (AMA) webinar here.
- The Log Analytics agent collects information from Windows or Linux hosts. In addition to OS events such as Windows Events, the agent can collect events stored in files. Learn more about the Log Analytics agent in this blog: collecting telemetry from on-prem and IaaS server using the Log Analytics agent. The Azure Monitor Agent is a new generation agent currently in preview that offers advantages such as Windows events filtering. The legacy Log Analytics agent will be retired on 31 August 2024. We recommend that you keep up to date with the new features being released for the AMA over time, as it reaches towards parity with the MMA/OMS. Aim to migrate as soon as the features you need to run your Microsoft Sentinel deployment are available in the AMA. While you can run the MMA and AMA simultaneously, you may want to migrate each connector, one at a time, while running both agents.
- The Codeless Connector Platform (CCP) provides customers with the ability to create custom connectors, connect them, and ingest data to Microsoft Sentinel. Connectors created via the CCP can be deployed via API, an ARM template, or as a solution in the Microsoft Sentinel Content Hub.Connectors created using CCP are fully SaaS, without any requirements for service installations, and include health monitoring and full support from Microsoft Sentinel. Create a codeless connector for Microsoft Sentinel | Microsoft Docs provides guidelines and more information about this new way to connect data sources to Sentinel. To learn more about Codeless Connector platform watch the webinar: Youtube, Deck.
We now have a workbook to help navigate the complex JSON involved in deploying an ARM template for codeless connector platform (CCP) data connectors. Use the friendly interface of the codeless connector builder to simplify your development.
See our blog post for more details, Create Codeless Connectors with the Codeless Connector Builder (Preview)
- ELK's Logstash and Beats can be used as an alternative to both the agent and Log Forwarder using the Microsoft Sentinel Logstash output plug-in.
- Integrate Threat Intelligence (TI) sources using the built-in connectors from TAXII servers or Microsoft Graph Security API. Read more on how to in the documentation. TI can also be important as a custom log using a custom connector or as a lookup table. You can read more about how TI is used managed in Microsoft Sentinel in the TI modules later.
- Custom connectors use the ingestion API and therefore are similar to direct sources. Custom connectors are most often implemented using Logic Apps, offering a codeless option, or Azure Functions.
Module 5: Log Management
While how many and which workspaces to use is the first architecture question to ask, there are additional log management architectural decisions:
- Where and how long to retain data.
- How to best manage access to data and secure it.
Ingest, Archive, Search, and Restore Data within Microsoft Sentinel
Watch the webinar: Manage Your Log Lifecycle with New Methods for Ingestion, Archival, Search, and Restoration, here.
This suite of features contains:
- Basic ingestion tier: new pricing tier for Azure Log Analytics that allows for logs to be ingested at a lower cost. This data is only retained in the workspace for 8 days total.
- Archive tier: Azure Log Analytics has expanded its retention capability from 2 years to 7 years. With that, this new tier for data will allow for the data to be retained up to 7 years in a low-cost archived state.
- Search jobs: search tasks that run limited KQL in order to find and return all relevant logs to what is searched. These jobs search data across the analytics tier, basic tier. and archived data.
- Data restoration: new feature that allows users to pick a data table and a time range in order to restore data to the workspace via restore table.
- Learn more about this new feature here.
Alternative retention options outside of the Microsoft Sentinel platform
- If you want to retain data for more than two years or reduce the retention cost, you can consider using Azure Data Explorer for long-term retention of Microsoft Sentinel logs: Webinar Slides, Webinar Recording, Blog).
- Need more depth? Watch the Improving the Breadth and Coverage of Threat Hunting with ADX Support, More Entity Types, and Updated MITRE Integration webinar here.
- If you prefer another long-term retention solution, export from Microsoft Sentinel / Log Analytics to Azure Storage and Event Hub or move Logs to Long-Term Storage using Logic Apps. The latter advantage is that it can export historical data.
- Lastly, you can set fine-grained retention periods using table-level retention Settings (and documentation)
Log Security
- Use resource RBAC or table Level RBAC to enable multiple teams to use a single workspace.
- If needed, delete PII data from your workspaces.
- Learn how to audit workspace queries and Microsoft Sentinel use, using alerts workbooks and queries.
- Use private links to ensure logs never leave your private network.
Dedicated cluster
- More than 500GB per day? You can have your own Log Analytics dedicated cluster.
Module 6: Enrichment: TI, Watchlists, and more
One of the important functions of a SIEM is to apply contextual information to the event steam, enabling detection, alert prioritization, and incident investigation. Contextual information includes, for example, threat intelligence, IP intelligence, host and user information, and watchlists.
Microsoft Sentinel provides comprehensive tools to import, manage, and use threat intelligence. For other types of contextual information, Microsoft Sentinel provides Watchlists, as well as alternative solutions.
Threat Intelligence
Watch the Automate Your Microsoft Sentinel Triage Efforts with RiskIQ Threat
Intelligence webinar. YouTube. Deck.
Short on time? watch the Ignite session (28 Minutes)
Get Deeper? Watch the Webinar: YouTube, MP4, Presentation
Threat Intelligence is an important building block of a SIEM. Watch the Explore the Power of Threat Intelligence in Microsoft Sentinel webinar here
In Microsoft Sentinel, you can integrate threat intelligence (TI) using the built-in connectors from TAXII servers or through the Microsoft Graph Security API. Read more on how to in the documentation. Refer to the data collection modules for more information about importing Threat Intelligence.
Once imported, Threat Intelligence is used extensively throughout Microsoft Sentinel and is weaved into the different modules. The following features focus on using Threat Intelligence:
- View and manage the imported threat intelligence in Logs in the new Threat Intelligence area of Microsoft Sentinel.
- Use the built-in TI Analytics rule templates to generate security alerts and incidents using your imported threat intelligence.
- Visualize key information about your threat intelligence in Microsoft Sentinel with the Threat Intelligence workbook.
Watchlists and other lookup mechanisms
Watch the Use Watchlists to Manage Alerts, Reduce Alert Fatigue and improve
SOC efficiency webinar. YouTube, Deck.
Module W: Log transformation
Watch the Transforming Data at Ingestion Time in Microsoft Sentinel webinar. YouTube, Deck.
Get Deeper? Watch the Webinar: Manage Your Log Lifecycle with New Methods for Ingestion, Archival, Search, and Restoration. Youtube, Deck
Microsoft Sentinel supports two new features for data ingestion and transformation. These features, provided by Log Analytics, act on your data even before it's stored in your workspace.
The first of these features is the custom logs API. It allows you to send custom-format logs from any data source to your Log Analytics workspace, and store those logs either in certain specific standard tables, or in custom-formatted tables that you create. The actual ingestion of these logs can be done by direct API calls. You use Log Analytics data collection rules (DCRs) to define and configure these workflows.
The second feature is ingestion-time data transformation for standard logs. It uses DCRs to filter out irrelevant data, to enrich or tag your data, or to hide sensitive or personal information. Data transformation can be configured at ingestion time for the following types of built-in data connectors:
- AMA-based data connectors (based on the new Azure Monitor Agent)
- MMA-based data connectors (based on the legacy Log Analytics Agent)
- Data connectors that use Diagnostic settings
- Service-to-service data connectors
For more information, see:
Module X: Migration
Watch the Webinar: YouTube, MP4, Presentation
In many (if not most) cases, you already have a SIEM and need to migrate to Microsoft Sentinel. While it may be a good time to start over and rethink your SIEM implementation, it makes sense to utilize some of the assets you already built in your current implementation. To start watch our webinar describing best practices for converting detection rules from Splunk, QRadar, and ArcSight to Microsoft Sentinel Rules: YouTube, MP4, Presentation, blog.
Migrate to Microsoft Sentinel with the SIEM migration experience
The new Microsoft Sentinel Migration experience helps customers and partners automate the process of migrating their security monitoring use cases hosted in non-Microsoft products into Microsoft Sentinel.
- This first version of the tool supports migrations from Splunk
Watch the webinar on the Splunk to Microsoft Sentinel Migration Experience
You might also be interested in some of the resources presented in the blog:
Module Z: ASIM and Normalization
Watch the Extend and Manage ASIM: Developing, Testing and Deploying Parsers
webinar: YouTube, Deck.
Watch the Advanced SIEM Information Model (ASIM): Now built into Microsoft Sentinel webinar:
YouTube, Deck.
Working with various data types and tables together presents a challenge. You must become familiar with many different data types and schemas, write and use a unique set of analytics rules, workbooks, and hunting queries for each, even for those that share commonalities (for example, DNS servers). Correlation between the different data types necessary for investigation and hunting is also tricky.
The Advanced SIEM Information Model (ASIM) provides a seamless experience for handling various sources in uniform, normalized views. ASIM aligns with the Open-Source Security Events Metadata (OSSEM) common information model, promoting vendor agnostic, industry-wide normalization. Watch the Advanced SIEM Information Model (ASIM): Now built into Microsoft Sentinel webinar: YouTube, Deck.
The current implementation is based on query time normalization using KQL functions. And includes the following:
- Normalized schemas cover standard sets of predictable event types that are easy to work with and build unified capabilities. The schema defines which fields should represent an event, a normalized column naming convention, and a standard format for the field values.
- Watch the Understanding Normalization in Microsoft Sentinel webinar: Youtube, Presentation.
- Watch the Deep Dive into Microsoft Sentinel Normalizing Parsers and Normalized Content webinar: YouTube, MP3, Presentation.
- Parsers map existing data to the normalized schemas. Parsers are implemented using KQL functions. Watch the Extend and Manage ASIM: Developing, Testing and Deploying Parsers webinar: YouTube, Deck.
- Content for each normalized schema includes analytics rules, workbooks, hunting queries, and additional content. This content works on any normalized data without the need to create source-specific content.
Using ASIM provides the following benefits:
- Cross source detection: Normalized analytic rules work across sources, on-prem and cloud, now detecting attacks such as brute force or impossible travel across systems including Okta, AWS, and Azure.
- Allows source agnostic content: the coverage of built-in as well as custom content using ASIM automatically expands to any source that supports ASIM, even if the source was added after the content was created. For example, process event analytics support any source that a customer may use to bring in the data, including Defender for Endpoint, Windows Events, and Sysmon. We are ready to add Sysmon for Linux and WEF once released!
- Support for your custom sources in built-in analytics
- Ease of use: once an analyst learns ASIM, writing queries is much simpler as the field names are always the same.
To learn more about ASIM:
- Watch the overview webinar: YouTube, slides.
- Watch the Deep Dive into Microsoft Sentinel Normalizing Parsers and Normalized Content webinar: YouTube, MP3, Presentation.
- Watch the Turbocharging ASIM: Making Sure Normalization Helps Performance Rather Than Impacting It webinar: YouTube, MP4, Deck.
- Read the documentation.
To Deploy ASIM:
- Deploy the parsers from the folders starting with “ASIM*” in the parsers folder on GitHub.
- Activate analytic rules that use ASIM. Search for “normal” in the template gallery to find some of them. To get the full list use this GitHub search.
To Use ASIM:
- Use the ASIM hunting queries from GitHub
- Use ASIM queries when using KQL in the log screen.
- Write your own analytic rules using ASIM or convert existing ones.
- Write parsers for your custom sources to make them ASIM compatible and take part in built-in analytics
Part 3: Creating Content
What is Microsoft Sentinel's content?
Microsoft Sentinel security value is a combination of its built-in capabilities such as UEBA, Machine Learning, or out-of-the-box analytics rules and your capability to create custom capabilities and customize built-in ones. Customized SIEM capabilities are often referred to as "content" and include analytic rules, hunting queries, workbooks, playbooks, and more.
In this section, we grouped the modules that help you learn how to create such content or modify built-in-content to your needs. We start with KQL, the Lingua Franca of Microsoft Sentinel. The following modules discuss one of the content building blocks such as rules, playbooks, and workbooks. We wrap up by discussing use cases, which encompass elements of different types to address specific security goals such as threat detection, hunting, or governance.
Module 7: KQL
Short on time? Start at the beginning and go as far as time allows.
Most Microsoft Sentinel capabilities use KQL or Kusto Query Language. When you search in your logs, write rules, create hunting queries, or design workbooks, you use KQL. Note that the next section on writing rules explains how to use KQL in the specific context of SIEM rules.
We suggest you follow this Sentinel KQL journey:
- Pluralsight KQL course - the basics
- The Microsoft Sentinel KQL Lab: An interactive lab teaching KQL focusing on what you need for Microsoft Sentinel:
- Learning module (SC-200 part 4)
- Deck, Lab URL
- a Jupyter Notebooks version contributed by jjsantanna, which let you test the queries within the notebook.
- Learning webinar: Youtube, MP4;
- Reviewing lab solutions webinar: YouTube , MP4
- Microsoft Sentinel Workbooks 101 (with sample Workbook) blog.
- Pluralsight Advanced KQL course
- Optimizing Microsoft Sentinel KQL queries performance: YouTube, MP4, Deck.
- Using ASIM in your KQL queries: YouTube, Deck
- KQL Framework for Microsoft Sentinel -Empowering You to Become KQL-Savvy: YouTube, Deck.
You might also find the following reference information useful as you learn KQL:
Module 8: Analytics
Writing Scheduled Analytics Rules
Short on time? watch the Webinar: MP4, YouTube, Presentation
Microsoft Sentinel enables you to use built-in rule templates, customize the templates for your environment, or create custom rules. The core of the rules is a KQL query; however, there is much more than that to configure in a rule.
To learn the procedure for creating rules, read the documentation. To learn how to write rules, i.e., what should go into a rule, focusing on KQL for rules, watch the webinar: MP4, YouTube, Presentation.
SIEM rules have specific patterns. Learn how to implement rules and write KQL for those patterns:
- Correlation rules: using lists and the "in" operator or using the "join" operator
- Aggregation: see using lists and the "in" operator above, or a more advanced pattern handling sliding windows
- Lookups: Regular, or Approximate, partial & combined lookups
- Handling false positives
- Delayed events: are a fact of life in any SIEM and are hard to tackle. Microsoft Sentinel can help you mitigate delays in your rules.
- Using KQL functions as building blocks: Enriching Windows Security Events with Parameterized Function.
To blog post "Blob and File Storage Investigations" provides a step by step example of writing a useful analytic rule.
Using built-in analytics
Short on time? watch the Machine Learning Webinar: MP4, YouTube, Presentation
Before embarking on your own rule writing, you should take advantage of the built-in analytics capabilities. Those do not require much from you, but it is worthwhile learning about them:
- Use the built-in scheduled rule templates. You can tune those templates by modifying the templates the same way to edit any scheduled rule. Make sure to deploy the templates for the data connectors you connect listed in the data connector "next steps" tab.
- Learn more about Microsoft Sentinel's Machine learning capabilities: MP4, YouTube, Presentation
- Find the list of Microsoft Sentinel's Advanced multi-stage attack detections ("fusion") are enabled by default.
- Watch the Fusion ML Detections with Scheduled Analytics Rules webinar: YouTube, MP4, Presentation.
- Learn more about Microsoft Sentinel's built-in SOC-ML anomalies here.
- Watch the customized SOC-ML anomalies and how to use them webinar here: YouTube, MP4, Deck.
- Watch the Fusion ML Detections for Emerging Threats & Configuration UI webinar here: YouTube, Deck.
Module 9: Implementing SOAR
Watch the What’s New in Microsoft Sentinel Automation webinar:
YouTube, MP4, Deck
In modern SIEMs such as Microsoft Sentinel, SOAR (Security Orchestration, Automation, and Response) comprises the entire process from the moment an incident is triggered and until it is resolved. This process starts with an incident investigation and continues with an automated response. The blog post "How to use Microsoft Sentinel for Incident Response, Orchestration and Automation" provides an overview of common use cases for SOAR.
Automation rules are the starting point for Microsoft Sentinel automation. They provide a lightweight method for central automated handling of incidents, including suppression, false-positive handling, and automatic assignment.
To provide robust workflow based automation capabilities, automation rules use Logic App playbooks:
- Watch the Unleash the automation Jedi tricks & build Logic Apps Playbooks like a Boss Webinar: YouTube, MP4, Deck
- Read about Logic Apps, which is the core technology driving Microsoft Sentinel playbooks.
- The Microsoft Sentinel Logic App connector is a link between Logic Apps and Microsoft Sentinel.
You can find dozens of useful Playbooks in the Playbooks folder on the Microsoft Sentinel GitHub, or read "A playbook using a watchlist to Inform a subscription owner about an alert" for a Playbook walkthrough.
While Microsoft Sentinel is a cloud-native SIEM, its automation capabilities do extend to on-prem environments, either using the Logic Apps on-prem gateway or using Azure Automation as described in "Automatically disable On-prem AD User using a Playbook triggered in Azure"
Module 10: Workbooks, reporting, and visualization
Short on time? Watch the Webinar: YouTube, MP4, Deck
Workbooks
As the nerve center of your SOC, you need Microsoft Sentinel to visualize the information it collects and produces. Use workbooks to visualize data in Microsoft Sentinel.
- To learn how to create workbooks, read the documentation or watch Billy York's Workbooks training (and accompanying text).
- Those resources are not Microsoft Sentinel specific, and apply to Microsoft Workbooks in general. To learn more about Workbooks in Microsoft Sentinel, watch the Webinar: YouTube, MP4, Deck, and read the documentation.
Workbooks can be interactive and enable much more than just charting. With Workbooks, you can create apps or extension modules for Microsoft Sentinel to complement built-in functionality. We also use workbooks to extend the features of Microsoft Sentinel. Few examples of such apps you can both use and learn from are:
- The Investigation Insights Workbook provides an alternative approach for investigating incidents.
- Graph Visualization of External Teams Collaborations enables hunting for risky Teams use.
- The users' travel map workbook allows investigating geo-location alerts.
- The insecure protocols workbook (Implementation Guide, recent enhancements, and overview video) let you identify the use of insecure protocols in your network.
- Lastly, learn how to integrate information from any source using API calls in a workbook.
You can find dozens of workbooks in the Workbooks folder in the Microsoft Sentinel GitHub. Some of those are available in the Microsoft Sentinel workbooks gallery and some are not.
Reporting and other visualization options
Workbooks can serve for reporting. For more advanced reporting capabilities such as reports scheduling and distribution or pivot tables, you might want to use:
- Power BI, which natively integrates with Log Analytics and Sentinel.
- Excel, which can use Log Analytics and Sentinel as the data source (and see a a video on how)
- Jupyter notebooks covered later in the hunting module are also a great visualization tool.
Module Y: Notebooks
Short on time? Watch the short introduction video
Go deeper? Watch the Azure Notebooks Fundamentals Webinar: YouTube, MP4, Presentation
Want to become a Notebooks Ninja? Watch the Webinars:
1. Getting Started - YouTube, MP4, Presentation
2. MSTICPy Fundamentals - YouTube
3. MSTICPy Intermediate - YouTube
Jupyter notebooks are fully integrated with Microsoft Sentinel. While usually considered an important tool in the hunter's tool chest and discussed the webinars in the hunting section below, their value is much broader. Notebooks can serve for advanced visualization, an investigation guide, and for sophisticated automation.
To understand them better, watch the Introduction to notebooks video. Get started using the Notebooks webinar (YouTube, MP4, Presentation) or by reading the documentation. The Microsoft Sentinel Notebooks Ninja series is an ongoing training series to upskill you in Notebooks.
An important part of the integration is implemented by MSTICPY, a Python library developed by our research team for use with Jupyter notebooks that adds Microsoft Sentinel interfaces and sophisticated security capabilities to your notebooks.
Module 11: Use cases and solutions
Watch the Create Your Own Microsoft Sentinel Solutions webinar - YouTube, Deck.
Want more? Watch the Microsoft Sentinel Content Management webinar - YouTube, Deck.
Using connectors, rules, playbooks, and workbooks enables you to implement use cases: the SIEM term for a content pack intended to detect and respond to a threat. You can deploy Sentinel built-in use cases by activating the suggested rules when connecting each Connector. A solution is a group of use cases addressing a specific threat domain.
The Webinar "Tackling Identity" (YouTube, MP4, Presentation) explains what a use case is, how to approach its design, and presents several use cases that collectively address identity threats.
Another very relevant solution area is protecting remote work. Watch our ignite session on protection remote work, and read more on the specific use cases:
- Microsoft Teams hunting use cases and Graph Visualization of External Microsoft Teams Collaborations
- Monitoring Zoom with Microsoft Sentinel: custom connectors, analytic rules, and hunting queries.
- Monitoring Windows Virtual Desktop with Microsoft Sentinel: use Windows Security Events, Azure AD Sign-in logs, Microsoft 365 defender for endpoints, and WVD diagnostics logs to detect and hunt for WVD threats.
- Monitor Microsoft endpoint Manager / Intune, using queries and workbooks.
- Microsoft Sentinel solution for Microsoft Power Platform, allows you to monitor and detect suspicious or malicious activities in your Power Platform environment. The solution collects activity logs from different Power Platform components and inventory data. For more information see: Security content reference, learn how to deploy the solution here. Watch the webinar on how to Secure Dynamics 365 and Power Platform w/ Microsoft Sentinel Solutions for MS Business Applications
And lastly, focusing on recent attacks, learn how to monitor the software supply chain with Microsoft Sentinel.
Microsoft Sentinel solutions provide in-product discoverability, single-step deployment, and enablement of end-to-end product, domain, and/or vertical scenarios in Microsoft Sentinel. Read more about them here, and watch the webinar about how to create your own here. For more about Sentinel content management in general, watch the Microsoft Sentinel Content Management webinar - YouTube, Deck.
Part 4: Operating
Module 12: Handling incidents
Recommended content:
Watch the Announcing the New Microsoft Sentinel Incident Investigation Experience! webinar: Youtube
Go deeper? Decrease Your SOC'S MTTR (Mean Time to Respond) by Integrating Microsoft
Sentinel with Microsoft Teams webinar here: YouTube, Deck.
Short on time? Watch the "day in a life" Webinar: YouTube, MP4, Deck
After building your SOC, you need to start using it. The "day in a SOC analyst life" webinar (YouTube, MP4, Presentation) walks you through using Microsoft Sentinel in the SOC to triage, investigate and respond to incidents.
Integrating with Microsoft Teams directly from Microsoft Sentinel enables your teams to collaborate seamlessly across the organization, and with external stakeholders. Watch the Decrease Your SOC’s MTTR (Mean Time to Respond) by Integrating Microsoft Sentinel with Microsoft Teams webinar here.
You might also want to read the documentation article on incident investigation. As part of the investigation, you will also use the entity pages to get more information about entities related to your incident or identified as part of your investigation.
Incident investigation in Microsoft Sentinel extends beyond the core incident investigation functionality. We can build additional investigation tools using Workbooks and Notebooks (the latter are discussed later, under hunting). You can also build additional investigation tools or modify ours to your specific needs. Examples include:
- The Investigation Insights Workbook provides an alternative approach for investigating incidents.
- Notebooks enhance the investigation experience. Read 'Why Use Jupyter for Security Investigations?" and learn how to investigate with Microsoft Sentinel & Jupyter Notebooks: part 1, part 2, and part 3.
Incident tasks, which help you standardize your incident investigation and response practices so you can more effectively manage incident workflow, are now generally available (GA) in Microsoft Sentinel.
Get deeper on Incident tasks with:
- Use tasks to manage incidents in Microsoft Sentinel
- Work with incident tasks in Microsoft Sentinel
- Audit and track changes to incident tasks in Microsoft Sentinel
See this blog post by that shows how you can use incident tasks in combination with watchlists, automation rules, and playbooks to build a task management solution.
Module 13: Hunting
Short on time? watch the webinar Workbooks Deep Dive: YouTube, MP4, Deck
(Note that the Webinar starts with an update on new features, to learn about hunting, start at slide 12. The YouTube
link is already set to start there)
Go deeper? Watch the webinar What’s New in Microsoft’s Jupyter and Python Security Toolset: Youtube
While most of the discussion so far focused on detection and incident management, hunting is another important use case for Microsoft Sentinel. Hunting is a proactive search for threats rather than a reactive response to alerts.
The hunting dashboard was recently refreshed in July 2021 and shows all the queries written by Microsoft's team of security analysts and any extra queries that you have created or modified. Each query provides a description of what it hunts for, and what kind of data it runs on. These templates are grouped by their various tactics - the icons on the right categorize the type of threat, such as initial access, persistence, and exfiltration. Read more about it here.
To understand more about what hunting is and how Microsoft Sentinel supports it, Watch the hunting intro Webinar (YouTube, MP4, Deck). Note that the Webinar starts with an update on new features. To learn about hunting, start at slide 12. The YouTube link is already set to start there.
While the intro webinar focuses on tools, hunting is all about security. Our security research team webinar on hunting (MP4, YouTube, Presentation) focuses on how to actually hunt. The follow-up AWS Threat Hunting using Sentinel Webinar (MP4, YouTube, Presentation) really drives the point by showing an end-to-end hunting scenario on a high-value target environment. Lastly, you can learn how to do SolarWinds Post-Compromise Hunting with Microsoft Sentinel and WebShell hunting motivated by the latest recent vulnerabilities in on-premises Microsoft Exchange servers.
Module 14: User and Entity Behavior Analytics (UEBA)
Watch Present and Future of Users Entity Behavioral Analytics in Sentinel
webinar: YouTube, Deck.
Short on time? Watch the Webinar: YouTube, Deck , MP4
Microsoft Sentinel newly introduced User and Entity Behavior Analytics (UEBA) module enables you to identify and investigate threats inside your organization and their potential impact - whether a compromised entity or a malicious insider.
Learn more about UEBA in the UEBA Webinar (YouTube, Deck, MP4) and read about using UEBA for investigations in your SOC.
Enrichment widgets in Microsoft Sentinel
Enrichment widgets are dynamic components that provide you with in-depth, actionable intelligence about entities. They integrate external and internal content and data from various sources, allowing you a better understanding of potential security threats.
- Learn how to enable enrichment widgets in this article
- Learn more about enabling enrichment widgets and SOC applications in this blog post
Module 15: Monitoring Microsoft Sentinel's health
Short on time? watch the videos on monitoring connectors,
security operations health or workspace audit.
Part of operating a SIEM is making sure it works smoothly and an evolving area in Microsoft Sentinel. Use the following to monitor Microsoft Sentinel's health:
- Measure the efficiency of your Security operations (video)
- SentinelHealth data table. Provides insights on health drifts, such as latest failure events per connector, or connectors with changes from success to failure states, which you can use to create alerts and other automated actions. Find more information here.
- Monitor Data connectors health (video) and get notifications on anomalies
- Monitor agents using the agents' health solution (Windows only) and the Heartbeat table (Linux and Windows)
- Monitor your Log Analytics workspace: YouTube, MP4, Presentation, including query execution and ingest health
- Cost management is also an important operational procedure in the SOC. Use the Ingestion Cost Alert Playbook to ensure you are aware in time of any cost increase.
Part 5: Advanced Topics
Module 16: Extending and Integrating using Microsoft Sentinel APIs
Short on time? watch the video (5 minutes)
Get deeper? Watch the Webinar: MP4, YouTube, Presentation
As a cloud-native SIEM, Microsoft Sentinel is an API first system. Every feature can be configured and used through an API, enabling easy integration with other systems and extending Sentinel with your own code. If API sounds intimidating to you, don't worry; whatever is available using the API is also available using PowerShell.
To learn more about Microsoft Sentinel APIs, watch the short introductory video and blog post. To get the details, watch the deep dive Webinar (MP4, YouTube, Presentation) and read the blog post Extending Microsoft Sentinel: APIs, Integration, and management automation.
Module 17: Bring your own ML
Short on time? watch the video
Microsoft Sentinel provides a great platform for implementing your own Machine Learning algorithms. We call it Bring Your Own ML or BYOML for short. Obviously, this is intended for advanced users. If you are looking for built-in behavioral analytics, use our ML Analytic rules, UEBA module, or write your own behavioral analytics KQL based analytics rules.
To start with bringing your own ML to Microsoft Sentinel, watch the video, and read the blog post. You might also want to refer to the BYOML documentation.
** Azure Sentinel became Microsoft Sentinel in Nov 2021. Although effort has been made to update the name throughout the ninja training, some webinars and presentations may still refer to Azure Sentinel rather than Microsoft Sentinel as they were created and recorded before the name change. **
