Microsoft 365 Compliance audit log activities via O365 Management API - Part 1

Published Nov 12 2021 09:35 AM 16.2K Views
Microsoft

Auditing and reporting play important roles in the security and compliance strategy for many organizations. With the continued expansion of the technology landscape that has an ever-increasing number of systems, endpoints, operations, and regulations, it becomes even more important to have a comprehensive logging and reporting solution in place.

 

Auditing and reporting benefits

There are two main benefit categories for continued auditing and reporting, those are:

 

Security benefits

Audit logs can be used to detect security violations and/or breaches, after or – sometimes – before they occur. They can also be used to identify internal organization policy violations or misuse of systems or information. More importantly, logs can be used to assist with recovery processes (either from an attack or a system outage) or troubleshooting technical problems. Finally, audit logs are valuable assets and strong evidence in legal battles, which can save organizations endless headaches.

 

Compliance benefits

Many organizations need to comply with various regulations to be able to operate in a certain business (Finance, healthcare, government, etc.). one of the main requirements of these regulatory bodies is keeping (and reviewing!) logs and reports. Audit logs can be considered during an audit to prove that the organization complies with regulations.

The National Institute of Standards and Technology (NIST) has published an interesting article that discusses in great detail the how and the why you should use audit logs.

 

Microsoft 365 Auditing Solutions

Microsoft 365 auditing solutions provide an integrated solution to help organizations effectively respond to security events, forensic investigations, internal investigations, and compliance obligations. Thousands of user and admin operations performed in dozens of Microsoft 365 services and solutions are captured, recorded, and retained in your organization's unified audit log. Audit records for these events are searchable by security ops, IT admins, insider risk teams, and compliance and legal investigators in your organization. This capability provides visibility into the activities performed across your Microsoft 365 organization.

Microsoft 365 auditing capabilities address customers' needs and pain points:

  • The need to know who accessed what, when, and how across Microsoft 365 workloads.
  • Gain visibility into changes and modifications to policies, configurations, and sensitive information in Microsoft 365.
  • The need for investigation capabilities and tools to identify security breaches, risks, and compliance violations.

To learn more about Microsoft 365 auditing solutions, please check our comprehensive official documentation.

 

Microsoft 365 MIP/DLP Auditing Architecture

All the Microsoft 365 services send their log events via the “auditing pipeline” to centralized storage. In fact, this storage location is not one solution, it comprises multiple technologies (Azure tables, substrate, etc.) but we won’t go into details in this blog about these technologies. There are 3 main methods you can access/search/export Microsoft 365 services audit logs in your tenant as illustrated in the below diagram.

Walid_Elmorsy_0-1636682981638.jpeg

Figure 01 - Microsoft 365 Auditing Architecture for MIP and DLP event logs

 

  • Microsoft 365 Compliance Centre – Unified Audit Log: this is the main location (if an audit is enabled in the tenant). You can access the unified audit log via both GUI in the compliance center portal (as explained here in detail) and PowerShell (as explained here in detail) to search and export logs.
  • Microsoft 365 Activity Explorer: Activity Explorer allows you to monitor what is being done with your labeled content by providing a historical view of activities on this labeled content. The activity information is collected from the Microsoft 365 unified audit logs, transformed, and made available in the Activity explorer UI. Activity explorer reports on up to 30 days’ worth of data. To learn more about activity explorer, visit Microsoft’s official documentation here.
  • Office 365 Management APIs: The Office 365 Management APIs provide a single extensibility platform for all Office 365 management tasks, including service communications, security, compliance, reporting, and auditing. All the Office 365 Management APIs are consistent in design and implementation with the current suite of Office 365 REST APIs, using common industry-standard approaches, including OAuth v2, OData v4, and JSON. Like the other Office 365 APIs, applications are registered in Azure Active Directory, giving developers a consistent way to authenticate and authorize their apps. To learn more about Office 365 management APIs, visit Microsoft’s official documentation here.

 

Office 365 Management API

The Office 365 Management Activity API aggregates actions and events into tenant-specific content blobs, which are classified by the type and source of the content they contain. Currently, these content types are supported:

  • Audit.AzureActiveDirectory
  • Audit.Exchange
  • Audit.SharePoint
  • Audit.General (includes all other workloads not included in the previous content types)
  • DLP.All (DLP events only for all workloads)

 

Note:

DLP sensitive data is only available in the activity feed API to users that have been granted “Read DLP sensitive data” permissions.

 

For detailed information about Management Activity API, please check the official reference here.

 

MIP/DLP in M365 Unified Audit log

Let’s start by exploring what MIP and DLP activities are captured in Microsoft 365 Unified Audit log. There are various activities captured from different M365 services (as referenced here). There are also other factors to consider while trying to locate the audited activity you are after, such as:

  • What workload the activity was performed on? Such as EXO, SPO, etc.
  • Where was the activity performed from? Thick Office client vs. Office Web App.

Below are the main 5 categories we will be looking at in this article in terms of MIP-related user activities (specifically sensitivity label activities) captured in M365 Audit log across applications (i.e., Thick and Web office clients). Please note that there are also several MIP/DLP-related administrator activities (such as policy configuration changes, etc.) that are captured in the audit log as well, however we will be focusing on the user activities in this article.:

  • Applied Sensitivity Label (emails, files, and containers)
  • Changed/Updated Sensitivity Label (emails, files, and containers)
  • Removed Sensitivity Label (emails, files, and containers)
  • Sensitivity Labelled File Opened (files)
  • Sensitivity Labelled File Renamed (files)

Note:

For MIP in Power BI activities’ overview and audit log schema, please check the Microsoft official documentation here and here are the best place to stay on top of the changes.

 

MIP-Activities Audit Log Schema

For each captured Audit log, there are properties (i.e., schema parameters) attached to it to detail all the activities, sources, actions, etc. for that log. Let’s try to go over some of the audit log schema parameters for email and file events to make it clearer. Please note that the below is a cut-down list of the schema as most of the schema parameters are self-explanatory. For full and updated list of log schema, please visit Microsoft’s official documentation here. For a full list of Microsoft 365 audited activities, visit here.

 

Note:

The log schema fields and parameters in the below paragraphs and tables are always updating. As the workload evolves, now workloads are introduced or updated, the log fields are subject to change or update. We will do our best to keep them updated and refreshed. The two reference official articles documentation here and here are the best place to stay on top of the changes.

 

Base Schema (Common)

The base schema section of the table below contains the majority of the common parameters of MIP audit logs for both emails and files.

Email log event example

As an example, below is an M365 Audit log event that was captured that represents applying a sensitivity label to an email message:

 

{"CreationTime":"2021-10-27T05:22:05","Id":"9901d727-9448-4ad8-8375-8333c22452fa","Operation":"SensitivityLabelApplied","OrganizationId":"-8385-8539b47e6810","RecordType":83,"UserKey":"fe461d1e-0173-426f-b0eb-107556e69982","UserType":0,"Version":1,"Workload":"PublicEndpoint","ClientIP":"x.y.z.113","UserId":"admin@nodomain.xyz","Application":"Outlook","DeviceName":"xyz-AADC01","Platform":"Windows","UserSku":"NoSKUsRetrieved","EmailInfo":{"from":"admin@nodomain.xyz","subject":"test","to":["walid.elmorsy@nodomain.com"]},"TargetLocation":4,"SensitivityLabelEventData":{"ActionSource":3,"LabelEventType":1,"SensitivityLabelId":"ec8371e3-f2aa-4136-bf5b-fa4ad15ccb98"}}

 

File log event example

As an example, below is an M365 Audit log event that was captured for updating (i.e., changing) a sensitivity label on an office file in SharePoint Online:

 

{"AppAccessContext":{"CorrelationId":"255af89f-4086-1000-0e7d-951e4133fb5d"},"CreationTime":"2021-10-12T07:44:13","Id":"6c006638-c5cd-4329-60d1-08d98d5415d8","Operation":"FileSensitivityLabelChanged","OrganizationId":"123456-8385-8539b47e6810","RecordType":6,"UserKey":"i:0h.f|membership|xyz123@live.com","UserType":0,"Version":1,"Workload":"SharePoint","ClientIP":"1.2.3.31","ObjectId":"https:\/\/NoDomain.sharepoint.com\/sites\/SalesAndMarketing\/Shared Documents\/not important.docx","UserId":"user.one@NoDomain.xyz","CorrelationId":"255af89f-4086-1000-0e7d-951e4133fb5d","EventSource":"SharePoint","ItemType":"File","ListId":"a5ad9657-7b94-4cb1-b8ac-436ca76592ce","ListItemUniqueId":"43a9437c-5b1a-4ede-82a7-fc977967a0a1","Site":"f984326a-338a-4659-91cb-a8d6bc2974eb","UserAgent":"MSWAC","WebId":"0caa4d59-7887-4240-899a-7a2372db5ba1","DestinationFileExtension":"docx","HighPriorityMediaProcessing":false,"SensitivityLabelEventData":{"ActionSource":3,"OldSensitivityLabelId":"eb1626ba-828c-4d6a-86e7-246ac348994b","SensitivityLabelId":"08bb4979-ee3b-4a1d-aaed-a8e32694d5a8","OldSensitivityLabelOwnerEmail":"admin@NoDomain.xyz","SensitivityLabelOwnerEmail":"user.one@NoDomain.xyz"},"SourceFileExtension":"docx","DestinationFileName":"not important.docx","DestinationLabel":"08bb4979-ee3b-4a1d-aaed-a8e32694d5a8","DestinationRelativeUrl":"sites\/SalesAndMarketing\/Shared Documents","SiteUrl":"https:\/\/NoDomain.sharepoint.com\/sites\/SalesAndMarketing\/","SourceFileName":"not important.docx","SourceLabel":"eb1626ba-828c-4d6a-86e7-246ac348994b","SourceRelativeUrl":"Shared Documents"}

 

MIP log event schema reference for emails and files

 

Schema parameter

Description

Potential Values

Notes

Base Schema – MIP for Email and/or File

RecordType

The operation type indicated by the record

  • 06 = SharePointFileOperation
  • 43 = MIPLabel
  • 48 = LabelContentExplorer
  • 71 = MipAutoLabelSharePointItem
  • 72 = MipAutoLabelSharePointPolicyLocation      
  • 75 = MipAutoLabelExchangeItem    
  • 82 = SensitivityLabelPolicyMatch
  • 83 = SensitivityLabelAction
  • 84 = SensitivityLabeledFileAction
  • 93 = AipDiscover (AIP Scanner event)
  • 109 = MipExactDataMatch

For a complete updated list and full description of the Log RecordType, please refer to this article. Here we are only listing the relevant MIP Record types.

UserType

The user type that performed the operation

  • 0 = Regular
  • 1 = Reserved
  • 2 = Admin
  • 3 = DcAdmin
  • 4 = Systeml
  • 5 = Application
  • 6 = ServicePrincipal
  • 7 = CustomPolicy
  • 8 = SystemPolicy

DcAdmin = A Microsoft datacentre operator

Application

The application where the activity occurred

  • Outlook (for email)
  • OWA (for email)
  • Word (for file)
  • Excel (for file)
  • PowerPoint (for file)

 

Platform

The platform where the activity occurred from

  • 0 = Unknown
  • 1 = Windows
  • 2 = MacOS
  • 3 = iOS
  • 4 = Android
  • 5 = Web Browser

In the log, the “Platform” parameter can either have the platform called out (i.e., Windows as per the example above), or the numeric value (i.e., 5 as per below):

Walid_Elmorsy_0-1636684251259.png

 

Workload

The workload where the activity occurred on

  • Exchange
  • SharePoint
  • OneDrive
  • MicrosoftTeams
  • PublicEndPoint

 

DeviceName

The device where the activity occurred

  • Device name (Outlook)
  • Outlook.office.com (OWA)

Device FQDN will be populated if the activity occurred on a thick client (i.e., Outlook), otherwise it will be (outlook.office.com) if occurred from OWA

Activity

The activity Type for the audit log

  • Applied Sensitivity Label
  • Updated Sensitivity Label
  • Removed Sensitivity Label

 

Operation

The operation type for the audit log (Referenced here as discussed above)

  • SensitivityLabelApplied
  • SensitivityLabelUpdated
  • SensitivityLabelRemoved
  • SensitivityLabelPolicyMatched
  • SensitivityLabeledFileOpened
  • SensitivityLabeledFileRenamed
  • SensitivityLabelApplied is applicable to: Manual/Default/Recommended label: Only for brand new email messages.
  • SensitivityLabelUpdated is applicable to: Manual/Recommended: Only for label change activities in forward and replies (i.e., not brand-new email messages)
  • SensitivityLabelPolicyMatched is applicable to “recommended” label.

TargetLocation

The location of the Email.

  • 0 = Unknown
  • 1 = LocalMedia
  • 2 = RemovableMedia
  • 3 = Fileshare
  • 4 = Cloud

This will always be (4 – Cloud) for email messages, since they are stored in the EXO mailboxes.

Email/MIP-Specific Extended Schema (the EmailInfo and SensitivityLabelEventData sections in the log)

PolicyID

The Auto-Label policy ID (if the email was auto-labelled)

  • Auto-label Policy ID

Only application (and visible in the logs) for auto-labelled emails.

UserID

The sender SMTP address (delegated scenario)

  • SMTP address of the sender/owner (for non-delegated mailbox)
  • SMTP address of the mailbox delegate (for delegated mailbox)

The SMTP address of the sender on-behalf of the mailbox (for the delegated mailbox scenario).

ActionSource

This field indicates whether the label was applied manually or automatically

  • 0 = None
  • 1 = Default
  • 2 = Auto
  • 3 = Manual
  • 4 = Recommended

The value “None” covers operation types such as “CopiedToUSB”.

ActionSourceDetail

This field list the “details” of the ActionSource

  • 0 = None
  • 1 = AutoByPolicyMatch
  • 2 = AutoByReplyOrForward
  • 3 = AutoByInheritance
  • 4 = AutoByDeploymentPipeline
  • 5 = PublicAPI
  • 6 = AutoByLibraryDefault

 

LabelEventType

 

  • 0 = None
  • 1 = LabelUpgraded
  • 2 = LabelDowngraded
  • 3 = LabelRemoved
  • 4 = LabelChangedSameOrder

 

SensitivityLabelID

The current MIP sensitivity label GUID

  • Label GUID

Applies to both brand-new emails and/or as the “New label GUID” for label change activities for manual/auto label activities.

OldSensitivityLabelID

The previous MIP sensitivity label GUID

  • Label GUID

Only applies to emails with label change activities (Upgrade or Downgrade label) for manual/auto label activities.

JustificationText

The justification provided by the user when the sensitivity label is downgraded or removed.

  • Predefined “justification text”
  • Other: “Custom justification text provided by the user”

 

File-Specific Extended Schema

ObjectId

The entity on which the activity was performed.

  • File full path (URL)

For SharePoint and OneDrive for Business activity, the full path name of the file or folder accessed by the user.

IrmContentId

The unique ID used for identifying the encrypted document after the operation is complete

  • irmContentID associated with the document (if any)

 

 

 

In the next part of this blog series, we will continue discussing the DLP audit log schema, where can we find the Microsoft 365 compliance MIP & DLP related logs in the Office 365 Management API content blobs, as well as configuring the prerequisites and sharing a script to query Office 365 Management API. Moreover we will explore how the exported log data can be used, and we will show an example using PowerBI to draft a report.

 

 

References

 

Additional resources

MIP & Compliance One Stop Shop Resource Page: https://aka.ms/mipc/OSS

Read all the latest MIP updates and blogs at: https://aka.ms/MIPblog

Join MIP & Compliance preview programs at: https://aka.ms/MIPC/Previews

 

Thank you.

6 Comments
Co-Authors
Version history
Last update:
‎Jan 31 2022 04:39 PM
Updated by: