Hi, all!
Welcome to a new series on Microsoft Sentinel Notebooks! In this post, we want to introduce everyone to the Notebooks feature of Microsoft Sentinel and provide some basic knowledge that we’ll build on throughout this series.
The series will take the following form:
- Part 1: What are notebooks and when do you need them? – this post
- Part 2: How to get started with notebooks and tour of the features
- Part 3: Overview of the pre-built notebooks and how to use them
- Part 3.5: Using Code Snippets to build your own Sentinel Notebooks
- Part 4: How to create your own notebooks from scratch and how to customize the existing ones
KNOWLEDGE CHECK: And, once you've completed all of the parts of this series, you can take the Knowledge Check. If you score 80% or more in the Knowledge Check, you can expect your very own Notebooks Ninja participation certificate from us.
The diagram below demonstrates a structured learning pathway for you to become a Microsoft Sentinel Notebooks ninja and earn a Ninja certificate.
What are notebooks and when to use them?
We have a number of features built into Microsoft Sentinel that share the “books” nomenclature, i.e., Playbooks, Workbooks, and Notebooks – so it can be confusing at times.
Playbooks, of course, are based on Azure Logic Apps and supply some of the automation capabilities for Microsoft Sentinel. Workbooks are provided for analysts and SOC managers to build interactive views and reports of the Sentinel data.
Notebooks should be an integral part of the security team’s daily processes, particularly those security teams using Microsoft Sentinel as their SIEM of choice.
The Notebooks feature in Microsoft Sentinel is built on Jupyter Notebooks which is an open-source web application that allows anyone to create and share documents that contain live code, equations, visualizations, and narrative text. Its name is derived from the scripting language it is based on: JUlia, PYThon, and R.
The Jupyter Notebooks service has gained its popularity in various data science and scientific computing communities such as Genome research, Astronomy, finance, and stock market predictions, among others. It's effective and reliable proficiency to dynamically parse and present data enabled a logical pathway of interest to the cybersecurity field and has increasingly become a key tool for cybersecurity operations.
In Why Use Jupyter for Security Investigations?, @ianhelle provides some great context for using Jupyter Notebooks for cybersecurity operations including the capability for accessing and including external data, providing a true scripting and programming environment, and providing a set of steps that are restartable and repeatable.
Think of a notebook like OneNote on steroids. Just like OneNote, you can store valuable information like text and pictures, but in the case of notebooks, that data is interactive.
In short…
- Notebooks can be artifact storage – data persistence, repeatability and backtracking allows analysts to collect and store evidence and collateral to improve response the next time a similar event occurs.
- Notebooks can be interactive – storing more than just pieces of information, notebooks can process the scripts it stores and produce data results inline to be used to build a more efficient and more intelligent approach to investigations and hunting.
- Notebooks can be interoperable – Notebooks enable deeper programmatic abilities to connect to, store, and use external data to be used dynamically.
- Notebooks can be guides – through sophisticated data processing, machine learning, and visualization, notebooks guide analysts through every step of an investigation or hunt to expose, mitigate, and remediate threats to the environment.
Notebooks in Microsoft Sentinel extend the capabilities of the overall product. Out-of-the-box, every Microsoft Sentinel instance comes with several ready-made notebooks that provide use cases for things like:
- Scanning for credential leaks in your database environment
- Detecting malicious base64-encoded commands on Linux hosts
- Generating baselines in network activities
To have a better understanding of who needs to use which “book” and when, the following table breaks these areas down side-by-side per suggested role along with providing the uses and pros and cons for each.
|
Playbooks |
Workbooks |
Notebooks |
|
|
Roles |
• SOC engineers • Analysts of all tiers |
• SOC engineers • Analysts of all tiers • SOC managers |
• Threat hunters/Tier 2-3 analysts • Incident investigators • Cyber data scientists • Security researchers |
|
Uses |
Automation of simpler, repeatable tasks:
|
|
|
|
Pros |
|
|
|
|
Cons |
• Not suitable for ad-hoc & complex chain of tasks • Not great for documenting & sharing evidence |
• Cannot integrate with external data |
• Higher learning curve - requires coding knowledge * • Limited automated execution (automation capabilities should be improved in the near future) |
* Anyone can use our built-in notebooks without coding knowledge. But the additional skillsets are involved in taking notebooks to an advanced level. This is one of the reasons for the effort behind this Microsoft Sentinel Notebook Ninja series, but also a big reason for an upcoming public-facing, free training series for Microsoft Sentinel Notebooks.
To register visit https://aka.ms/SecurityWebinars, look for Microsoft Sentinel | Become a Notebooks ninja webinar and fill out the registration form.
We are super-excited to be bringing this series (and the training) to you! Look for more great knowledge on Microsoft Sentinel Notebooks as we supply new installments of this series.
Additionally, we've launched a brand new email DL specifically for Microsoft Sentinel Notebooks: asinotebooks@service.microsoft.com. This DL is monitored by various product teams and is intended to be used to collect and respond to questions, issues, and feedback.
Stay tuned!
More reading/tutorial resources:
- Notebooks at the Azure Sentinel GitHub Repository
- Use Jupyter Notebook to hunt for security threats
- Detect Credential Leaks using Azure Sentinel Notebooks! (video)
- Azure Sentinel: Notebooks - Getting Started (video)
- Detect Malicious Base64-Encoded Commands on Linux Hosts (video)
- Microsoft Sentinel Weekly Newsletter
Special thanks to my dear colleague, @Rod_Trent, for his major collaboration on this blog series and drafting this post!
