Becoming a Microsoft Sentinel Notebooks Ninja - The Series!

Published Aug 27 2021 11:12 AM 11.5K Views
Microsoft

Hi, all!

 

Welcome to a new series on Microsoft Sentinel Notebooks!  In this post, we want to introduce everyone to the Notebooks feature of Microsoft Sentinel and provide some basic knowledge that we’ll build on throughout this series.

 

The series will take the following form:

 

The diagram below demonstrates a structured learning pathway for you to become an Microsoft Sentinel Notebooks ninja and earn a Ninja certificate.

 

Notebook ninja training series.png

 

What are notebooks and when to use them?

 

We have a number of features built into Microsoft Sentinel that share the “books” nomenclature, i.e., Playbooks, Workbooks, and Notebooks – so it can be confusing at times.

 

Playbooks, of course, are based on Azure Logic Apps and supply some of the automation capabilities for Microsoft Sentinel. Workbooks are provided for analysts and SOC managers to build interactive views and reports of the Sentinel data.

 

Notebooks should be an integral part of the security team’s daily processes, particularly those security teams using Microsoft Sentinel as their SIEM of choice.

 

The Notebooks feature in Microsoft Sentinel is built on Jupyter Notebooks which is an open-source web application that allows anyone to create and share documents that contain live code, equations, visualizations, and narrative text. Its name is derived from the scripting language it is based on: JUlia, PYThon, and R.

 

The Jupyter Notebooks service has gained its popularity in various data science and scientific computing communities such as Genome research, Astronomy, finance, and stock market predictions, among others. It's effective and reliable proficiency to dynamically parse and present data enabled a logical pathway of interest to the cybersecurity field and has increasingly become a key tool for cybersecurity operations.

 

In Why Use Jupyter for Security Investigations?, @ianhelle  provides some great context for using Jupyter Notebooks for cybersecurity operations including the capability for accessing and including external data, providing a true scripting and programming environment, and providing a set of steps that are restartable and repeatable.

 

Think of a notebook like OneNote on steroids. Just like OneNote, you can store valuable information like text and pictures, but in the case of notebooks, that data is interactive.

 

In short…

  • Notebooks can be artifact storage – data persistence, repeatability and backtracking allows analysts to collect and store evidence and collateral to improve response the next time a similar event occurs.
  • Notebooks can be interactive – storing more than just pieces of information, notebooks can process the scripts it stores and produce data results inline to be used to build a more efficient and more intelligent approach to investigations and hunting.
  • Notebooks can be interoperable – Notebooks enable deeper programmatic abilities to connect to, store, and use external data to be used dynamically.
  • Notebooks can be guides – through sophisticated data processing, machine learning, and visualization, notebooks guide analysts through every step of an investigation or hunt to expose, mitigate, and remediate threats to the environment.

 

Notebooks in Microsoft Sentinel extend the capabilities of the overall product. Out-of-the-box, every Microsoft Sentinel instance comes with several ready-made notebooks that provide use cases for things like:

 

 

To have a better understanding of who needs to use which “book” and when, the following table breaks these areas down side-by-side per suggested role along with providing the uses and pros and cons for each.

 

Playbooks

Workbooks

Notebooks

Roles

•       SOC engineers

•       Analysts of all tiers

•       SOC engineers

•       Analysts of all tiers

•       SOC managers

•       Threat hunters/Tier 2-3 analysts

•       Incident investigators

•       Cyber data scientists

•       Security researchers

Uses

Automation of simpler, repeatable tasks:

  • Ingestion – bring in external data
  • Enrichment (TI, GeoIP lookups, etc.)
  • Investigation
  • Remediation
  • Visualization
  • Querying Microsoft Sentinel & external data 
  • Enrichment (TI, GeoIP, WhoIs lookups, etc.)
  • Investigation
  • Visualization
  • Hunting
  • Machine Learning & big data analytics

Pros

  • Best for single, repeatable tasks
  • No coding knowledge required
  • Best for high-level view of Sentinel data
  • No coding knowledge required
  • Best for more complex chain of repeatable tasks
  • Ad-hoc, more procedural control – easy to pivot due to the interactive characteristics and the use of Python, a procedural language
  • Rich Python libraries for data manipulation & visualization options
  • Machine Learning & custom analysis
  • Easy to document & share analysis evidence

Cons

•       Not suitable for ad-hoc & complex chain of tasks

•       Not great for documenting & sharing evidence

•   Cannot integrate with external data

•       Higher learning curve - requires coding knowledge *

•       Limited automated execution (automation capabilities should be improved in the near future)

 

* Anyone can use our built-in notebooks without coding knowledge. But the additional skillsets are involved in taking notebooks to an advanced level. This is one of the reasons for the effort behind this Microsoft Sentinel Notebook Ninja series, but also a big reason for an upcoming public-facing, free training series for Microsoft Sentinel Notebooks.

 

To register visit https://aka.ms/SecurityWebinars, look for Microsoft Sentinel | Become a Notebooks ninja webinar and fill out the registration form.

 

We are super-excited to be bringing this series (and the training) to you! Look for more great knowledge on Microsoft Sentinel Notebooks as we supply new installments of this series.

 

Additionally, we've launched a brand new email DL specifically for Microsoft Sentinel Notebooks: asinotebooks@service.microsoft.com. This DL is monitored by various product teams and is intended to be used to collect and respond to questions, issues, and feedback.

 

Stay tuned!

 

More reading/tutorial resources:

 

Special thanks to my dear colleague, @rodtrent, for his major collaboration on this blog series and drafting this post!

 

 

1 Comment
Version history
Last update:
‎Nov 18 2021 04:44 AM
Updated by: