Azure AD Mailbag: Discovering and blocking legacy authentication

I know this article is pretty old but I thought I might drop a note here.

I've tried doing this very same thing in my own tenant to try and stop spray attacks that are almost always sourced by IMAP4. I have a policy constructed similarly above but configured to allow legacy only from our public IP address but block any other attempts at legacy/basic auth. I have a pilot group and when I use the "What If" function in conditional access for those users outside of our IP it comes back and shows that it will block connection. However, when I look our sign in logs spray attacks are still happening against those users even with the policy active and the "What If" tool confirming the connection should be blocked.

This is absolutely maddening and the MS Support folks say "See, it is working because the attempt was blocked" but it wasn't blocked. My condition policy was not applied and the only reason it failed is because they hit the threshold for failed login attempts.

AndresCanello