Data Loss Prevention (DLP) policies not working or inconsistent

[Alex Melching]

Hello Community,

I have been struggling with simple DLP policies and can't get them to respond as advertised.  I have a very simple one that I used the pre-built template for to detect US SSNs in location email only.  I set it to detect, show policy tip, and require user to provide justification to be able to send.  Published the policy and waited several hours. In testing, I can get it to detect a SSN if I use 111111111 and then type in ssn or something close to it as suggested in of the Office Support doc for pattern matching.  Well, some times it works and others no.  It's very inconsistent when in a draft email. And it's very odd when using different combinations of the SSN form with or without dashes or spaces.  I will receive the policy tip in Outlook on the web, but never in the Outlook client.  The timing of policy tip is odd as well, sometimes it can be as quick as 5 seconds or as long as 5 minutes.  Also, why does it not detect on number pattern match alone, it must have some combination of text for SSN to trigger.

I have even tweaked the Min and Max settings as suggested in another Office Support doc, but am met with the same experience as above.  I have worked with Microsoft Support and still have opening ticket with them about this and they are as baffled as I am.  And I can reproduce the same experience in several Office 365 tenants.

Any help or recommendations to much appreciated.  I would like to use the policy tips and justification, but now just stuck detecting it after it's sent and encrypting it.

Thanks!

[#2]

Usually you need this corroborative evidence to trigger a match, as the sensitive types are defined with low confidence level for just the SSN match. Of course if you create your own custom sensitive types, you can configure them as you wish. 

As for the policy tip, it's shown when the message is saved in OWA. Outlook needs to download the policy file before it's able to display tips, which might take a while.

[#3]

@Vasil Michev 

Hi, thanks for the response. I have tested tweaking the Min and Max to lower values, but the inconsistency just makes it useless.  I have not tried custom sensitivity types yet and will give it a whack.  

Are you saying that the policy tip in Outlook on the web will not display until the "new" message is saved as a draft?

The policy has been published for some time now and should have reached users' Outlook clients by now. Is there a way to confirm that or tell?

[#4]

Scan in OWA is performed every time the message is saved as Draft, and only then the policy tips will be displayed. Outlook uses its own method, generally speaking you should check the policy* xml files in %USERPROFILE%\Appdata\Local\Microsoft\Outlook. To force Outlook to redownload the xml files, delete the HKCU\Software\Microsoft\Office\16.0\Outlook\PolicyNudges key.

[#5]

@Vasil Michev 

Thanks for the great info.  That is super helpful in understanding that process for Outlook on the web.  I'm checking out the Outlook client files to.  

Giant thanks!