📘 Preface
This post demonstrates one method to export Dynamics 365 Customer Engagement (CE) Dataverse organization data using the Office 365 Management Activity API and Azure Functions. It is feasible for customers to build a custom lake-house architecture with this feed, enabling advanced analytics, archiving, or ML/AI scenarios.
🧭 When to Use This Custom Integration
While Microsoft offers powerful native integrations like Dataverse Synapse Link and Microsoft Fabric, this custom solution is observed implemented and relevant in the following scenarios:
- Third-party observability and security tools already use this approach
Solutions such as Splunk and other enterprise-grade platforms commonly implement integrations based on the Office 365 Management Activity API to ingest tenant-wide audit data. This makes it easier for customers to align with existing observability pipelines or security frameworks.
- Customers opt out of Synapse Link or Fabric
Whether due to architectural preferences, licensing constraints, or specific compliance requirements, some customers choose not to adopt Microsoft’s native integrations. The Office Management API offers a viable alternative for building custom data export and monitoring solutions tailored to their needs.
🎯 Why Use the Office 365 Management Activity API?
- Tenant-wide Data Capture: Captures audit logs and activity data across all Dataverse orgs in a tenant.
- Integration Flexibility: Enables export to Cosmos DB, cold storage, or other platforms for analytics, compliance, or ML/AI.
- Third-party Compatibility: Many enterprise tools use similar mechanisms to ingest and archive activity data.
🏗️ Architecture Overview
- Azure Function App (.NET Isolated): Built as webhook, processes notifications, fetches audit content, and stores filtered events in Cosmos DB.
- Cosmos DB: Stores audit events for further analysis or archiving.
- Application Insights: Captures logs and diagnostics for troubleshooting.
🛠️ Step-by-Step Implementation
1. Prerequisites
- Azure subscription
- Dynamics 365 CE environment (Dataverse)
- Azure Cosmos DB account (SQL API)
- Office 365 tenant admin rights
- Enable Auditing in Dataverse org
2. Register an Azure AD App
- Go to Azure Portal > Azure Active Directory > App registrations > New registration
- Note:
-
- Application (client) ID
- Directory (tenant) ID
- Create a client secret
- Grant API permissions:
-
- ActivityFeed.Read
- ActivityFeed.ReadDlp
- ServiceHealth.Read
- Grant admin consent
3. Set Up Cosmos DB
- Create a Cosmos DB account (SQL API)
- Create:
- Database: officewebhook
- Container: dynamicsevents
- Partition key: /tenantId
- Note endpoint URI and primary key
4. Create the Azure Function App
- Use Visual Studio or VS Code
- Create a new Azure Functions project (.NET 8 Isolated Worker)
- Add NuGet packages:
- Microsoft.Azure.Functions.Worker
- Microsoft.Azure.Cosmos
- Newtonsoft.Json
Function Logic:
- Webhook validation
- Notification processing
- Audit content fetching
- Event filtering
- Storage in Cosmos DB
5. Configure Environment Variables
{
"OfficeApiTenantId": "<your-tenant-id>",
"OfficeApiClientId": "<your-client-id>",
"OfficeApiClientSecret": "<your-client-secret>",
"CrmOrganizationUniqueName": "<your-org-name>",
"CosmosDbEndpoint": "<your-cosmos-endpoint>",
"CosmosDbKey": "<your-cosmos-key>",
"CosmosDbDatabaseId": "officewebhook",
"CosmosDbContainerId": "dynamicsevents",
"EntityOperationsFilter": {
"incident": ["create", "update"],
"account": ["create"]
}
}
6. Deploy the Function App
- Build and publish using Azure Functions Core Tools or Visual Studio
- Restart the Function App from Azure Portal
- Monitor logs via Application Insights
🔔 How to Subscribe to the Office 365 Management Activity API for Audit Notifications
To receive audit notifications, you must first subscribe to the Office 365 Management Activity API. This is a two-step process:
1. Fetch an OAuth2 Token
Authenticate using your Azure AD app credentials to get a bearer token:
# Define your Azure AD app credentials
$tenantId = "<your-tenant-id>"
$clientId = "<your-client-id>"
$clientSecret = "<your-client-secret>"
# Prepare the request body for token fetch
$body = @{
grant_type = "client_credentials"
client_id = $clientId
client_secret = $clientSecret
scope = "https://manage.office.com/.default"
}
# Fetch the OAuth2 token
$tokenResponse = Invoke-RestMethod -Method Post -Uri "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token" -Body $body
$token = $tokenResponse.access_token
2. Subscribe to the Content Type
Use the token to subscribe to the desired content type (e.g., Audit.General):
$contentType = "Audit.General"
$headers = @{
Authorization = "Bearer $token"
"Content-Type" = "application/json"
}
$uri = "https://manage.office.com/api/v1.0/$tenantId/activity/feed/subscriptions/start?contentType=$contentType"
$response = Invoke-RestMethod -Method Post -Uri $uri -Headers $headers
$response
⚙️ How the Azure Function Works
🔸 Trigger
The Azure Function is triggered by notifications from the Office 365 Management Activity API. These notifications include audit events across your entire Azure tenant—not just Dynamics 365.
🔸 Filtering Logic
Each notification is evaluated against your business rules:
- Organization match
- Entity type (e.g., incident, account)
- Operation type (e.g., create, update)
These filters are defined in the EntityOperationsFilter environment variable:
{
"incident": ["create", "update"],
"account": ["create"]
}
🔸 Processing
- If the event matches your filters, the function fetches the full audit data and stores it in Cosmos DB.
- If not, the event is ignored.
🔍 Code Explanation: The Run Method
1. Webhook Validation
string validationToken = query["validationToken"];
if (!string.IsNullOrEmpty(validationToken)) {
await response.WriteStringAsync(validationToken);
response.StatusCode = HttpStatusCode.OK;
return response;
} 2. Notification Handling
var notifications = JsonConvert.DeserializeObject<dynamic[]>(requestBody);
foreach (var notification in notifications)
{
if (notification.contentType == "Audit.General" && notification.contentUri != null)
{
// Process each notification
}
}
3. Bearer Token Fetch
string bearerToken = await GetBearerTokenAsync(log);
if (string.IsNullOrEmpty(bearerToken)) continue;
4. Fetch Audit Content
var requestMsg = new HttpRequestMessage(HttpMethod.Get, contentUri);
requestMsg.Headers.Authorization = new AuthenticationHeaderValue("Bearer", bearerToken);
var result = await httpClient.SendAsync(requestMsg);
if (!result.IsSuccessStatusCode) continue;
var auditContentJson = await result.Content.ReadAsStringAsync();
5. Deserialize and Filter Audit Records
var auditRecords = JsonConvert.DeserializeObject<dynamic[]>(auditContentJson);
foreach (var eventData in auditRecords) {
string orgName = eventData.CrmOrganizationUniqueName ?? "";
string workload = eventData.Workload ?? "";
string entityName = eventData.EntityName ?? "";
string operation = eventData.Message ?? "";
if (workload != "Dynamics 365" && workload != "CRM" && workload != "Power Platform") continue;
if (!entityOpsFilter.ContainsKey(entityName)) continue;
if (!entityOpsFilter[entityName].Contains(operation)) continue;
// Store in Cosmos DB
}
6. Store in Cosmos DB
var cosmosDoc = new {
id = Guid.NewGuid().ToString(),
tenantId = notification.tenantId,
raw = eventData
};
var partitionKey = (string)notification.tenantId;
var resp = await cosmosContainer.CreateItemAsync(cosmosDoc, new PartitionKey(partitionKey));
7. Logging and Error Handling
log.LogInformation($"Stored notification in Cosmos DB for contentUri: {notification.contentUri}, DocumentId: {cosmosDoc.id}");
catch (Exception dbEx) {
log.LogError($"Error storing notification in Cosmos DB: {dbEx.Message}");
}
🧠 Conclusion
This solution provides a robust, extensible pattern for exporting Dynamics 365 CE Dataverse org data to Cosmos DB using the Office 365 Management Activity API. Solution architects can use this as a reference for building or evaluating similar integrations, especially when working with third-party archiving or analytics solutions.
Microsoft