When applying DLP policies such as SSN in Exchange Online, it actually needs to have SSN in the email to trigger the message tip to show. Just the fact that the actual SSN numbers show up in the email should be triggering the message tip. Same logic applies for bank routing numbers and other forms of PII.
When using this policy to protect the firm and educate staff through the message tip and trigger the new Encrypt Only option, requiring 'SSN' to be in the email is not logical. The DLP policy will still trigger the mail rule, but the message tip shouldn't require the additional logic.
Currently with the DLP system we have in place we bounce back clear text emails to staff that have sensitive info and require them to put some thought into sending the email by putting a keyword in the subject or clicking an encrypt button. With message tips we are hoping to be proactive in notifying them of the sensitivity without staff having to go back and find the sent item and resend.