What am I looking at? - Using Notebooks to gain situational awareness.
Published Oct 07 2019 10:00 AM 4,736 Views

Being a SOC analyst is a demanding job, and one of the challenges that analysts need to overcome is building and maintaining a wide breadth of knowledge. When we think about a good analyst, we often think of the knowledge required in a purely technical sense, network protocol understanding, log analysis, or host forensics. However, there are some less technical elements that an analyst needs to also have at hand.  For instance, the organizational context surrounding a given alert: who is the user involved, where does the machine sit on the network, do the IP addresses make sense? This contextual knowledge plays a large role when triaging alerts and investigating threats and can turn a seemingly innocuous logon event into a major incident.  

With Azure Sentinel’s integration with Azure Notebooks we have an ideal platform to give some structure to collecting and analyzing this contextual data. Notebooks provide a flexible platform from which multiple, different data sets can be flexibly and easily queried, and where that data can be effectively combined with others to provide a more complete picture.  By using Notebooks in this way you can save analysts critical time and effort when triaging alerts and provide a consistent, repeatable process that even a brand new analysts can pick up and use. 

I am not going to look at the basics of Notebooks – if you want this my colleague @ianhelle  has done some great blogs (1,2,3,4) on this topic. Instead I want to show you some approaches you can use to develop your own context building Notebook. 


Collecting the contextual data on a user 

One of the most valuable sources of information is often Active Directory (AD), it provides us with a rich source of data on users, devices, and organization. With many organizations moving rapidly to the cloud this information often resides in Azure Active Directory (AAD)Whilst there are many ways to get data out of AAD one of the best resources available is the Microsoft Graph API which exposes not just data from AAD but a whole host of Microsoft cloud services. To make leveraging this API even easier is the fact that our Python library for defendersmsticpy, includes prebuilt connectors for the Graph. 

Getting an access token for the Graph APIGetting an access token for the Graph API

Simply establish a Query Provider for the SecurityGraph environment and then pass it the credentials of your AAD app configured to query the Graph. This in turn will return you an access token for future API requests. 

Once connected you can pass API queries via the Query Provider you established, get the results and display them. 

 Collecting user information from the GraphCollecting user information from the Graph

 There are a wide range of items you can query the Graph for including user images, manager details, and telephone numbers.  You can customize what you collect, and display based on need or user preference. Some people respond better to visual imagery than textual data so for them returning a user image might be more insightful than their name.  

Displaying a user’s image from the GraphDisplaying a user’s image from the Graph

Collecting the contextual data on a host 

Contextual host data can be just as insightful as user data, understanding what role a host plays in the environment, who’s responsibility it is, and where it is located can make a big impact on an analysts assessment of a situation. Luckily in most cases this data is even easier to collect than the user data from the Graph.  

When collecting log data from Windows or Linux hosts in Azure Sentinel the preferred method is to install the OMS agent to perform this collection. As part of the agent operation it sends back heartbeats from the host, confirming connectivity and data collection is proceeding as expected. As part of this heartbeat data is a set of contextual information regarding the host itself which is stored in the Heartbeat data of your Azure Sentinel Workspace. 

In addition Azure Service Map is a service that maps services running within your Azure environment and builds up a picture of services, connections and dependencies. By enabling that service, collecting the logs in Azure Sentinel and querying it we can get further detail on the services running on a host. 

Collecting and displaying host dataCollecting and displaying host data

 However, this may not be where you choose to store your authoritative host details, or it might not provide the whole picture of a host. Many organizations store such data in an IT Service Management (ITSM) tool such as ServiceNow. The great thing about Notebooks is they can do virtually anything you can program them do, so with a few lines of code we can query our ServiceNow instance’s API and collect host data from there as well. The same approach could be used with any ITSM that has an exposed API to query details from. 

Collecting ticket information from ServiceNow for a hostCollecting ticket information from ServiceNow for a host

Collecting the contextual data on an IP address 

It doesn’t have to be an external data that you use to add context, even knowing whether or not an IP address is private or public, or is one you own or not can be vital information. Most experienced SOC analysts will know the RFC 1918 ranges like the back of their hand, however a more junior analyst may not be 100% sure on this yet, or a new team member might not know which public ranges you own. By writing a simple elif statement we can produce an easy to understand output for any analyst. Providing details on an IP addressProviding details on an IP address

There are also a wide range of external data sources that can add context to an IP address as well, from threat intelligence feeds, to whois data, DNS records, to more ad-hoc data such as lists ToR exit nodes lists. By using the flexibility of Notebookswe can collect and present this data as needed to add context in a format that is easily to understand quickly. Checking if an IP address appears in a list of known ToR exit nodesChecking if an IP address appears in a list of known ToR exit nodes


Notebooks provide the flexibility SOC teams need to collect context required during investigations quickly and easily. We have seen some examples of how to do this here with a just a few lines of code but there are many other sources of data or methods that could be used to collate and present context. Once you have identified the data and sources pertinent to your organization you can build shareable, repeatable Notebooks, code blocks, or even code libraries to allow analysts to quickly collect and analyze the context they need to have situational awareness. Allowing them to focus on the threat and make a judgement on it faster and more effectively. 

For a more in depth look at this topic check out the recording of our recent webinar that included a section on this - https://techcommunity.microsoft.com/t5/Azure-Sentinel/Azure-Sentinel-Webinar-Recordings/m-p/845065 

Or to look at the Notebook that these examples are drawn from you can access it here - https://notebooks.azure.com/peter-bryan/projects/sentinel-webinar

1 Comment
Version history
Last update:
‎Mar 23 2021 04:44 PM
Updated by: