Azure Sentinel: Collecting logs from Microsoft Services and Applications

[Ofer_Shezaf]

This is part of a series of blogs on connectors. You might find what you are looking for also here:

• Syslog, CEF, Logstash and other 3rd party connectors grand list
• Azure Sentinel Agent: Collecting telemetry from on-prem and IaaS server
• Creating Custom Connectors

In this blog post:

• The Azure Monitor collection framework
• How to connect Azure resources to Azure Sentinel
• Understanding the Azure monitor schema
• Collecting from specific Microsoft and Azure sources 

Azure Sentinel supports collecting telemetry from a wide array of Microsoft sources. Some of them are listed in the Sentinel's connector page and documentation. However, Sentinel can collect logs from most Azure services and other Microsoft products, even when not listed above. 

The Azure Monitor collection framework

Azure Monitor, and its Log Analytics module, is the underlying log management platform powering Azure Sentinel. As such, any source that sends logs to Azure Monitor or Log Analytics supports inherently Azure Sentinel. Most Azure and Microsoft solutions support sending telemetry to Azure monitor. You can read more about Azure Monitor collection here: "Collect Azure platform logs in Log Analytics workspace in Azure Monitor."  

How can I collect from a supported Azure source?

The following provides a guide as to how to connect each resource using the portal to Log Analytics/Azure Sentinel. The actual portal flow may differ from resource to resource. 

To log a service to Sentinel, pick the service (1), select "Activity Log" from the menu (2), and then click the "Logs" button (3). Note that on this screen, before pressing "Logs," you can review the information that will be sent to Sentinel.

On the next screen, click "Add," then "Select workspace," and select the Sentinel workspace. In some cases, the service provides diagnostic telemetry but not audit logs. In such cases, use "Diagnostic settings" instead of "Activity Log" and select "Add diagnostic setting." Some sources do not use the method outlined above, and the instructions below would help,

Understanding and using the events

You can read more about the structure of the events received by Azure Monitor here. The telemetry may be stored in the AzureDiagnostics table or in a dedicated table depending on the mode used by the source.

Each event will include several standard fields such as time, Resource Id, and Tenant ID as described here, as well as per resource fields. Several standard fields available in each Log Analytics table and not just Azure resource tables such as TimeGenerated, Type, and billing information are listed here.

Also, you can find a full reference - still under construction - to the Azure Monitor table schema for all sources, not just Azure ones, here. The Azure Monitor GitHub contains queries and workbooks for many Azure services that can provide a starting point for understanding the logs sent by them.

The Big List

The bis list is now part of the grand list.

[jvaidya]

@Ofer_Shezaf 

Thanks for the post, quite informative. 

Could you please guide us how we can integrate Office365 Threat protection and Information governance alerts with Sentinel?

Also, it would be really good if playbook integration can be made available with Azure services analytic rules e.g. ASC, ATP Defender, MCAS and Identiy protection etc. 

Many thanks in advance,

[Ofer_Shezaf]

@jvaidya :

Private preview for Office 365 ATP alerts ingestion is starting very soon. Join our Private Previews program to get details.

As to playbooks: what is the workflow you look for? what should those playbooks do?

~ Ofer

[KrishhnaM]

@Ofer_Shezaf , thanks for the info...

And we are also looking for office 365 alert ingestion with sentinel, is there any update on that,

PS: I have tried ingesting office 365 alerts from management API using logic apps with reference to a blog in tech community, however the details of the alerts are majorly Missing.... 

[jvaidya]

@Ofer_Shezaf  Thanks for the reply. Glad to know private preview for Office365 alerts is starting soon, will get myself registered for the private preview.

e.g. Out of the box rules (Rules template) for these services can create incidents based on all alerts generated but these rules do not allow any integration of playbooks. Obviously, other option  which we are following is to create the scheduled query rules for these services alerts as well but as useful Entities information (IP, URL, Account Name etc.) are usually in description or other fields of alert mapping with Entities becomes bit challenging.

[Ofer_Shezaf]

@jvaidya : incident triggers, which will allow triggering playbooks for incidents of all types are expected to be in preview in a few weeks.

[SteveMiles70]

Great post @Ofer_Shezaf   Have shared on my LinkedIn Network.

[gd2020]

Can i Ingest logs from Azure Front Door into Sentinel 

[Riyad Amin]

Hello @Ofer_Shezaf 

Thanks for sharing.