Overrides and false positives in DLP policy end user experience

[Mike Rowland]

Ok so a user gets a policy applied to his/her document for let's say PCI compliance.

On the policy tip we give the user the option to override with a business justification or to report as a false positive.

If they click the "report" button in the policy tip where does that go? where do I as an admin go to review those and presumably take some kind of action on that report? allow and reclassify or keep the classification and inform the user.

I'd expect to see something in the S&C reports but I can't see a thing. I can view my overrides report and view where a user has overridden a classification but nothing anywhere else that lets me interact with any reported "cases"

[#2]

Should be in the DLP reports as detailed here: https://support.office.com/en-us/article/view-the-reports-for-data-loss-prevention-41eb4324-c513-4fa...

Are you saying you don't see the events, or the DLP reports altogether?

[#3]

Found it ( I think) - im missing DLP Insights.  I have the DLP report but I'm not seeing the warning triangles / insight icon. 

[#4]

The insights are fairly new addition, I don't have them in my tenant either.

[#5]

@Vasil Michevdo you know if there was an announcement about this new feature? I don't remember seeing one.

[#6]

Maybe I'm looking at the wrong thing? This isn't a new feature.

I get a policy tip for a DLP rule

I have the option to "report" my content as a false positive

where the dickens does that report button end up? 

I am expecting if there's a report button that somewhere I can go as a sec admin, view that report and either dismiss it and reply to the data owner or opt to reclassify and allow sharing.

There seems no information anywhere about this and no one at Microsoft seems to have a clue about it from what I can see.

If you have a report button then it must go somewhere or why have the button?

[#7]

I've had this issue too; you don't actually get report as far I am aware. However, if you go to the DLP page and click the graph called DLP false positives and overrides, you can then see a graph of both results and change it to the actual details. You can also request a report of this information which will appear in the reports area.

One thing to note is if you're not a domain admin and just compliance, you cannot schedule reports to come out automatically. Only a domain admin can do it and they only are visible to them (we tried this and my colleague only got the reports, not me; I have to get them manually).

Hope this helps

[#8]

"you can then see a graph of both results and change it to the actual details" - could you attach a screenshot of what you mean? My graph (and report) are both decidedly non interactive!!

[#9]

Hi Mike

Hopefully these help; if you click on Show details table, it will show the details of any overrides and false positives. Just to note that my screenshot currently doesn't as we have a ticket open with MS about policy tips not working on Outlook but okay on OWA (don't know if anyone else is experiencing this). 

Simon

[#10]

Thanks Simon!

I get that report but I can't interact with it? Are you able to?

As my point above I can see someone has reported something as a false positive but there seems to no way for an admin to say "ah ok, that's fine, I'll reclassify it and away you go"

Thanks for your help!

[#11 · Accepted solution]

No I'm not able to; I don't think you can.

If someone does put down it's a false positive and it's not, I usually go and speak to the individual or email them. There's no way that I know of to reclassify it.

I also have alerts turned on to me when people do it so when I get the email, it shows the override reason and false positive answers. If anyone puts anything that we don't agree with as being an acceptable answer, then we raise this with them/their line manager.

[#12]

I think that's the conclusion I'm coming to.

There is no way to actually do what I'm expecting - which I think would make total sense to be able to interact and deal with these incidents rather than having to go find a user and have a chat with them.

I have it set up to alert me and it sounds like that's the best I can hope for.

That's all I needed - no one was able to tell me if I was missing anything or not but you've got the same experience so sounds like it is what it is.

Thanks so much for your help!

[#13]

Sorry it wasn't the answer you were looking for. I agree, it's not the best system and would be great to reclassify the false positives. I have it set-up for NI numbers so would be great to reclassify dummy NI numbers so they get excluded, as that's where most of my false positives come from.

Glad I could help (a bit).

[#14]

Simon - you said "I also have alerts turned on to me when people do it so when I get the email, it shows the override reason and false positive answers. If anyone puts anything that we don't agree with as being an acceptable answer, then we raise this with them/their line manager. "

How did you configure DLP to get an email when someone overrides? I only see sending a weekly report to an email from the override chart  - is that what you are talking about?

[#15]

Hi Karen

In the Policy, under Editing Policy Settings, you can create advanced settings for your policy. One of the options is the following:

You can then specify the email address you want reports to and what should be contained in those reports.

Hope this helps

Simon

[#16]

That's the email for the policy match - right, I do that already, but that doesn't send a notification for the override and the justification the user put in. I thought you were alerted via email that a person used the override button and entered a justification. So far, I've only seen that appear in the override report which I can schedule to send me weekly. So besides that - there is nothing that tells you a user used the override right?

[#17]

Sorry Karen for any confusion, I wasn't very clear.

I've setup the DLP policy to alert me whenever someone does something that is against the policy. When they click the override, it will appear in that email, not in a separate email.

I've shared a redacted email of what that looks like but there is no separate email I'm afraid; just the main DLP policy incident report which can tell you what the employee did. Other than this and the report, I don't know of anything else to inform admins someone has clicked "override".

[#18]

Ok, so I see now that you see the override justification in an incident report when it's applied to Exchange, but I've got a DLP policy (with incident reports enabled) applied just to my OneDrive and am using the override from the OneDrive client - and I am actually not getting any incident reports when it's in OneDrive. I have CAS and setup a CAS alert policy - so I see it's triggering those, so I know it's happening - but now that I'm specifically looking for incident reports - I don't get them from OneDrive.

Do you get incident report emails like you showed that you got from an Exchange hit, but from a OneDrive match?

[#19]

I've had incident reports for OneDrive and SharePoint to flag files being uploaded (in the end they've been dummy data) but looking back at them, they don't show any override/justification - just the details of the file and who did it.