SOLVED

AIP Log Analytics duplicated records

Copper Contributor

Hi all,

 

Do you know why I have duplicates records on some of my scanned files in my Log Analytics? , the only difference I see is the LogId_g which is different.

 

Thank you.

4 Replies

@VoTran 

 

You have answered your own question, if there is a difference then it will be sent (even if the time/date is the same)

@CliveWatson 

 

Thank you Clive for the answer. But do you have an explanation why the scanned files would have another different logid_g if the AIP scanner has only scanned 1 time in this repository?

best response confirmed by VoTran (Copper Contributor)
Solution
Sorry I don't know this particular table, or what logid_g contains. Its possible that its an artefact of a retry, some logs do get multiple rows when there is a retry.

Hi @CliveWatson,

Like @VoTran I am also seeing duplicate rows in my Log Analytics workspace for AIP.  This is from an AIP Scanner job. Every file discovered by the scanner has a duplicate row.

Logid_g looks like some sort of GUID, but I cannot find a complete schema reference for InformationProtectionLogs_CL table (some of the columns are described here: https://docs.microsoft.com/en-us/azure/information-protection/reports-aip)

 

With the Distinct operator in KQL I've managed to de-dupe the rows for reporting, however it would be fantastic if we could get an explantation for the duplicates and complete schema reference if possible please?

1 best response

Accepted Solutions
best response confirmed by VoTran (Copper Contributor)
Solution
Sorry I don't know this particular table, or what logid_g contains. Its possible that its an artefact of a retry, some logs do get multiple rows when there is a retry.

View solution in original post