Incomplete data from Search-UnifiedAuditLog cmdlet for AzureAD record type

Copper Contributor

Hi,

 

From the below cmdlet I got AuditData parameter as an incomplete JSON string.

Search-UnifiedAuditLog -Operations 'Update User.' -RecordType azureactivedirectory -StartDate (Get-Date).AddDays(-30) -EndDate (Get-Date)

I attached the output which i got.

 

AuditData.PNG

Please help me with this case !!! 

 

22 Replies

Confirmed, I see the same. What's even worse, if you use the UI, you get a "Failure: Record truncated" error. I'm not sure how this made it to production, but it should be addressed ASAP. Open a support case.

As a workaround, you might be able to get the full event details from the Azure AD blade in the Azure portal.

I'm curious, what character length is it truncating at? I believe I am seeing something similar for which I posted a question for on github. To me it looked like it the JSON string was getting truncated at 3062 characters. If I get an answer there I will try and reply here as well! Link to the issue I created on github: https://github.com/MicrosoftDocs/office-docs-powershell/issues/1733

@Tony Redmond was chasing this up with some MS folks, perhaps he can share some info.

I'm still discussing the issue. Microsoft has accepted that a problem exists and they need to fix it. Stay tuned.

from the docs https://docs.microsoft.com/en-us/office365/securitycompliance/search-the-audit-log-in-security-and-c...

 

There's a 3,060-character limit for the data that's displayed in the AuditData field for an audit record. If the 3,060-character limit is exceeded, the data in this field is truncated.

The problem is not the documented character limit. It is an ingestion problem for specific events that causes the JSON payload to be truncated as the record is written. Engineering is working on the issue.

Great - at the end of the day I am hoping for a valid JSON output.  If individual fields have to be thrown away/truncated, so be it.

As I said, the truncation issue is being worked and we should have a solution soon. I am actively tracking the issue with engineering. See https://office365foritpros.com/2018/10/22/longer-retention-office365-auditdata/ 

Is that "soon" or "Microsoft soon™"? They sure are taking their sweet time with this...

The same problem is reproduceable for workload "CRM". Hopefully is Microsoft able to address this issue soon.

Hi,

 

I don't run the CRM workload... could you post an example here of a truncated record so that I can make sure that this workload is fixed in the work that's ongoing?

 

TR

I have modified the following sample (e.g. "CrmOrganizationUniqueName" was replaced with an dummy value and all GUID values have been replaced with "foobar")

 

 

{"CreationTime":"2018-11-10T20:00:14","Id":"foobar","Operation":"CrmDefaultActivity","OrganizationId":"foobar","RecordType":21,"ResultStatus":"Success","UserKey":"Unknown","UserType":2,"Version":1,"Workload":"CRM","ClientIP":"127.0.0.1","ObjectId":"Create email","UserId":"drt@alfapeople.com","CrmOrganizationUniqueName":"foobar","Fields":[{"Name":"subject","Value":"foobar"},{"Name":"description","Value":"foobar"},{"Name":"ownerid","Value":"foobar"},{"Name":"from","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"to","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"cc","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"regardingobjectid","Value":"foobar"},{"Name":"isworkflowcreated","Value":"False"},{"Name":"notifications","Value":"0"},{"Name":"followemailuserpreference","Value":"False"},{"Name":"readreceiptrequested","Value":"False"},{"Name":"foobar","Value":"False"},{"Name":"emailreminderstatus","Value":"0"},{"Name":"isemailfollowed","Value":"False"},{"Name":"emailremindertype","Value":"0"},{"Name":"isregularactivity","Value":"False"},{"Name":"deliveryreceiptrequested","Value":"False"},{"Name":"deliveryprioritycode","Value":"1"},{"Name":"isemailreminderset","Value":"False"},{"Name":"compressed","Value":"False"},{"Name":"prioritycode","Value":"1"},{"Name":"directioncode","Value":"True"},{"Name":"correlationmethod","Value":"0"},{"Name":"activityid","Value":"foobar"}],"InstanceUrl":"https:\/\/foobar.crm4.dynamics.com\/","ItemType":"Dynamics365","ItemUrl":"https:\/\/foobar.crm4.dynamics.com\/main.aspx?etn=email&pagetype=entityrecord&id=foobar","UserAgent":"","CorrelationId":"00000000-0000-0000-0000-000000000000","EntityId":"foobar","EntityName":"email","Message":"Create","PrimaryFieldValue":"","Query":"","QueryResults":"","ServiceContextId":"00000000-0000-0000-0000-000000000000","ServiceContextIdType":"","ServiceName":"Dynamics365","SystemUserId":"foobar","UserUp

Please find the sample below. I have replaced some values with an place-holder ("foobar")

 

{"CreationTime":"2018-11-10T20:00:14","Id":"foobar","Operation":"CrmDefaultActivity","OrganizationId":"foobar","RecordType":21,"ResultStatus":"Success","UserKey":"Unknown","UserType":2,"Version":1,"Workload":"CRM","ClientIP":"127.0.0.1","ObjectId":"Create email","UserId":"drt@alfapeople.com","CrmOrganizationUniqueName":"foobar","Fields":[{"Name":"subject","Value":"foobar"},{"Name":"description","Value":"foobar"},{"Name":"ownerid","Value":"foobar"},{"Name":"from","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"to","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"cc","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"regardingobjectid","Value":"foobar"},{"Name":"isworkflowcreated","Value":"False"},{"Name":"notifications","Value":"0"},{"Name":"followemailuserpreference","Value":"False"},{"Name":"readreceiptrequested","Value":"False"},{"Name":"foobar","Value":"False"},{"Name":"emailreminderstatus","Value":"0"},{"Name":"isemailfollowed","Value":"False"},{"Name":"emailremindertype","Value":"0"},{"Name":"isregularactivity","Value":"False"},{"Name":"deliveryreceiptrequested","Value":"False"},{"Name":"deliveryprioritycode","Value":"1"},{"Name":"isemailreminderset","Value":"False"},{"Name":"compressed","Value":"False"},{"Name":"prioritycode","Value":"1"},{"Name":"directioncode","Value":"True"},{"Name":"correlationmethod","Value":"0"},{"Name":"activityid","Value":"foobar"}],"InstanceUrl":"https:\/\/foobar.crm4.dynamics.com\/","ItemType":"Dynamics365","ItemUrl":"https:\/\/foobar.crm4.dynamics.com\/main.aspx?etn=email&pagetype=entityrecord&id=foobar","UserAgent":"","CorrelationId":"00000000-0000-0000-0000-000000000000","EntityId":"foobar","EntityName":"email","Message":"Create","PrimaryFieldValue":"","Query":"","QueryResults":"","ServiceContextId":"00000000-0000-0000-0000-000000000000","ServiceContextIdType":"","ServiceName":"Dynamics365","SystemUserId":"foobar","UserUp

Please find attached an sample of the audit log (value of "AuditData"), i have replaced some values with an place-holder ("foobar")

 

 

{"CreationTime":"2018-11-10T20:00:14","Id":"foobar","Operation":"CrmDefaultActivity","OrganizationId":"foobar","RecordType":21,"ResultStatus":"Success","UserKey":"Unknown","UserType":2,"Version":1,"Workload":"CRM","ClientIP":"127.0.0.1","ObjectId":"Create email","UserId":"drt@alfapeople.com","CrmOrganizationUniqueName":"foobar","Fields":[{"Name":"subject","Value":"foobar"},{"Name":"description","Value":"foobar"},{"Name":"ownerid","Value":"foobar"},{"Name":"from","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"to","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"cc","Value":"Microsoft.Xrm.Sdk.Entity[]"},{"Name":"regardingobjectid","Value":"foobar"},{"Name":"isworkflowcreated","Value":"False"},{"Name":"notifications","Value":"0"},{"Name":"followemailuserpreference","Value":"False"},{"Name":"readreceiptrequested","Value":"False"},{"Name":"foobar","Value":"False"},{"Name":"emailreminderstatus","Value":"0"},{"Name":"isemailfollowed","Value":"False"},{"Name":"emailremindertype","Value":"0"},{"Name":"isregularactivity","Value":"False"},{"Name":"deliveryreceiptrequested","Value":"False"},{"Name":"deliveryprioritycode","Value":"1"},{"Name":"isemailreminderset","Value":"False"},{"Name":"compressed","Value":"False"},{"Name":"prioritycode","Value":"1"},{"Name":"directioncode","Value":"True"},{"Name":"correlationmethod","Value":"0"},{"Name":"activityid","Value":"foobar"}],"InstanceUrl":"https:\/\/foobar.crm4.dynamics.com\/","ItemType":"Dynamics365","ItemUrl":"https:\/\/foobar.crm4.dynamics.com\/main.aspx?etn=email&pagetype=entityrecord&id=foobar","UserAgent":"","CorrelationId":"00000000-0000-0000-0000-000000000000","EntityId":"foobar","EntityName":"email","Message":"Create","PrimaryFieldValue":"","Query":"","QueryResults":"","ServiceContextId":"00000000-0000-0000-0000-000000000000","ServiceContextIdType":"","ServiceName":"Dynamics365","SystemUserId":"foobar","UserUp

 

Dear all,

Any news for that question ?

I tried to use the Web interface to export the data and discovered that AuditData field limitation truncated to 3000 chars
I created a dedicated PowerShell script using the special command:

 - Search-UnifiedAuditLog

 

And found the truncate is also done at this Powershell level, so when that issue will be fixed ?

 

Thanks for your feedback.

 

PS: 

I posted a script to manage that AuditLog:

 - https://techcommunity.microsoft.com/t5/Office-365/PowerShell-script-to-export-Audit-log-search-Data-...

The limitation still exist with the PS command

 

Fab

The problem still exists.

 

Microsoft applied an update to the code and the result is even worse than before. The audit records for Azure AD group operations now contain a lot of detail, but the audit data is badly terminated. The net result is that these events don't show up in the SCC.

 

Messages have been sent to Microsoft to ask if they can look at the issue again. It's sad, but this has been a problem that started in August 2018...

 

TR

Oh the wonders of the DevOps world...

I imagined that you'd like the current state of affairs...