MPullmann : According to the Microsoft docs for blocking basic auth https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/disable-basic-authentication-in-exchange-online#:~:text=When%20it%27s%20blocked%2C%20Basic%20authentication%20in%20Exchange%20Online%20is%20blocked%20at%20the%20first%20pre%2Dauthentication%20step%20(Step%201%20in%20the%20previous%20diagrams)%20before%20the%20request%20reaches%20Azure%20Active%20Directory , basic auth is blocked pre-authentication and never reaches the IDP (e.g. Azure AD), so that's probably why you can't see Azure sign-in log entries for failed requests. When I created an authentication policy on my tenant to block basic auth IMAP, that stopped logs for all the brute force/password spray (failed) attempts on IMAP in my logs. If you need to see the requests get logged, you probably have to re-enable basic auth temporarily with the blue Diag button above. The logs go back 30 days, so you could look at who connected in that time period to get an idea of what users were still using basic auth before it got disabled.