This installment is part of a broader learning series to help you become a Jupyter Notebook ninja in Microsoft Sentinel. The installments will be bite-sized to enable you to easily digest the new content.
- Part 1: What are notebooks and when do you need them?
- Part 2: How to get started with notebooks and tour of the features – this post
- Part 3: Overview of the pre-built notebooks and how to use them
- Part 3.5: Using Code Snippets to build your own Sentinel Notebooks
- Part 4: How to create your own notebooks from scratch and how to customize the existing ones
KNOWLEDGE CHECK: And, once you've completed all of the parts of this series, you can take the Knowledge Check. If you score 80% or more in the Knowledge Check, you can expect your very own Notebooks Ninja participation certificate from us.
Getting Started with Azure Sentinel Notebooks
As we discussed in Part 1 of this series, the Jupyter Notebook service is a powerful tool and an integral part of Azure Sentinel. It provides additional capability to help augment areas where Azure Sentinel may not scale as well.
Through our discussion with customers, we’ve noted that many have expressed interest in learning more about this topic. And most importantly, many want to know how to incorporate notebooks into the daily regimen to improve SOC workflows through enhanced investigation, threat hunting, and machine learning.
If you’ve never used Jupyter notebooks before it can feel daunting and seem a bit like a black box.
Many of our pre-built notebooks rely on a Python library called MSTICPy . Originally developed by Microsoft to support Jupyter Notebooks authoring for Azure Sentinel, MSTICPy (Microsoft Threat Intelligence Python Security Tools) is a Python library that addresses three primary requirements for security investigators and hunters: acquiring and enriching data, analyzing data, and visualizing data. MSTICPy serves to reduce the amount of code that would have to be written using other Python libraries that aren’t tailored for security. While Azure Sentinel on its own provides the ability to do much of the same, Jupyter Notebooks with MSTICpy provides deeper functionality in the following specific areas:
- Querying log data from multiple sources at once including Azure Sentinel and external sources like data lake, blob storage, third party providers, et al
- Enriching the data with Threat Intelligence, geolocations and Azure resource data
- Extracting Indicators of Activity (IoA) from logs and unpack encoded data
- Performing sophisticated analysis such as anomalous session detection and time series decomposition
- Visualizing data using interactive timelines, process trees and multi-dimensional Morph Charts
- Includes time-saving notebook tools such as widgets to set query time boundaries, select and display items from lists, and configure the notebook environment
Ideally, the best way to get started is to become comfortable with a few of the “quick start” notebooks that we’ve provided as part of the Azure Sentinel out-of-the-box experience. For our efforts in this blog post, we want to introduce you properly using the Getting Started Guide notebook that’s supplied in Azure Sentinel.
Understanding how MSTICPy fits into the scheme of the Azure Sentinel notebooks is important, as most Azure Sentinel notebooks start by initializing MSTICPy to define the minimum version for Python and MSTICPy, installing the latest version of MSTICPy if needed, and then running the init_notebook function. See the More reading/tutorial resources section at the bottom of this blog post for the steps to accomplish this and more.
Take a Tour
Our Docs teams are second-to-none. And, as such have provided some amazing guidance around using the Getting Started notebook, including running and initializing to adding threat intelligence and GeoIP provider settings to running queries to authenticating to your Azure Sentinel workspace from your notebook.
Please use the following article on our Docs platform to deliver a self-guided tour: Tutorial: Get started with Jupyter notebooks and MSTICPy in Azure Sentinel
We also have a video tutorial for Getting Started with Azure Sentinel ML Notebooks. This tutorial guides you through the basic steps of using notebooks for security analysis. It covers all the basic steps you need to understand to start using the notebooks provided with Azure Sentinel.
RECOMMENDED: After working through the “Getting Started” notebook to setup the Azure Sentinel Notebook environment, consider digging directly into the “A Tour of Cybersec notebook features” notebook. This notebook walks through some of the features of Azure Sentinel notebooks and MSTICPy. The notebook introduces the data queries, visualization, data analysis, enrichment with threat intelligence and pivot functions. It can be run against the Azure Sentinel workspace or run as standalone using sample data.
Notebooks, like all other components and features for Azure Sentinel, are under constant review and undergoing constant improvement. Improvements and changes come from feedback and suggestions from our customers. Since our first blog post in this series, we’ve created a special email DL just for Azure Sentinel Notebooks. You can use this DL to send your questions, issues, and feedback and our various product teams will monitor and respond. The DL to use is asinotebooks@service.microsoft.com
Don’t forget to sign up to attend the upcoming public-facing, free training series for Azure Sentinel Notebooks. For those that have already registered, the first session is scheduled for September 30, 2021. If you want to be included in additional training sessions, register using the form.
To register visit https://aka.ms/SecurityWebinars, look for Azure Sentinel | Become a Notebooks ninja webinar and fill out the registration form.
We are super-excited to be bringing this series (and the training) to you! Look for more great knowledge on Azure Sentinel Notebooks as we prepare to deliver Part 3 of this series: Overview of the pre-built notebooks and how to use them.
More reading/tutorial resources:
- Notebooks at the Azure Sentinel GitHub Repository
- Use Jupyter Notebook to hunt for security threats
- Detect Credential Leaks using Azure Sentinel Notebooks! (video)
- Azure Sentinel: Notebooks - Getting Started (video)
- Detect Malicious Base64-Encoded Commands on Linux Hosts (video)
- Microsoft Sentinel Weekly Newsletter
Microsoft
