Mac RDP client silent NTLM downgrade due to Kerberos realm case-sensitive check

While researching what it took to make Kerberos and the Protected Users group with the Mac RDP client, I found a nasty bug: when using the krb5.conf file for manual KDC configuration, the Mac RDP client silently downgrades to NTLM after receiving a successful AS-REQ from the KDC if the Kerberos realm casing doesn't match the one extracted from the username. I wrote a blog post on the topic with full details and steps: https://awakecoding.com/posts/mac-rdp-client-kerberos-and-protected-users-guide/

Let me summarize the issue, with my test environment:

• IT-HELP-DC.ad.it-help.ninja as my domain controller
• IT-HELP-TEST.ad.it-help.ninja as my destination RDP server
• ad.it-help.ninja as my Active Directory DNS domain name
• IT-HELP as my NetBIOS domain name (try hard not to use it)
• ad.it-help.ninja as my Kerberos realm (same as DNS domain name)
• ProtectedUser@ ad.it-help.ninja, a user that is a member of the Protected Users group

Here is my krb5.conf file that registers my realm twice: once in lowercase, once in uppercase:

[libdefaults]
    dns_lookup_kdc = false
    dns_lookup_realm = false

[realms]
    ad.it-help.ninja = {
        kdc = tcp/IT-HELP-DC.ad.it-help.ninja
    }
    AD.IT-HELP.NINJA = {
        kdc = tcp/IT-HELP-DC.ad.it-help.ninja
    }

[domain_realm]
    .ad.it-help.ninja = ad.it-help.ninja

The only single-realm registration that partially works is with the uppercase realm name, using the uppercase realm name in the username. In other words, ProtectedUser@ AD.IT-HELP.NINJA works, but not ProtectedUser@ ad.it-help.ninjamailto:email address removed for privacy reasons-help.ninja, with the following krb5.conf file:

[libdefaults]
    dns_lookup_kdc = false
    dns_lookup_realm = false

[realms]
    AD.IT-HELP.NINJA = {
        kdc = tcp/IT-HELP-DC.ad.it-help.ninja
    }

[domain_realm]
    .ad.it-help.ninja = ad.it-help.ninja

And if I register only the lowercase realm name, then Kerberos simply won't work with either ProtectedUser@ AD.IT-HELP.NINJA or ProtectedUser@ ad.it-help.ninjamailto:email address removed for privacy reasons-HELP.NINJA:

[libdefaults]
    dns_lookup_kdc = false
    dns_lookup_realm = false

[realms]
    ad.it-help.ninja = {
        kdc = tcp/IT-HELP-DC.ad.it-help.ninja
    }

[domain_realm]
    .ad.it-help.ninja = ad.it-help.ninja

While this may seem odd, I believe the problem is due to the fact that the KDC always sends back the Kerberos realm in uppercase in the AS-REP message. This means that even if the client sends as.it-help.ninja or AD.IT-HELP.NINJA in the AS-REQ, the AS-REP coming back from the KDC will always use the uppercase variant (AD.IT-HELP.NINJA).

While this difference in casing doesn't affect the kinit command in macOS, and that a Kerberos ticket can be successfully obtained, there appears to be a case sensitive comparison done after receiving the AS-REP to compare the Kerberos realm name that was sent with the one that was received. The area of code dealing with calls the system GSSAPI library, and if I found the right function, it would be using a call to memcmp() to compare bytes instead of a case-insensitive string comparison method, explaining the observed behavior.

Please use a similar krb5.conf file where DNS-based detection is disabled to avoid having DNS SRV detection somehow making it work as case insensitive. I don't really know what the DNS SRV detection does under the hood, but I suspect it may end up register both the lowercase and uppercase realm names internally. When crafting a krb5.conf file manually, I doubt many users will think of registering it twice by hand, especially when Wireshark and kinit both show a successful AS-REQ/AS-REP exchange.

In order to confirm the NTLM downgrade, I suggest adding a test user to the Protected Users group, or disabling inbound NTLM by GPO in the server, otherwise it will silently, and successfully, downgrade to NTLM without anything to let you know it happened. I confirmed this without a doubt by decrypting the RDP TLS traffic in Wireshark, and it definitely downgrades to NTLM when the issue occurs. You can find my tooling to decrypt TLS RDP traffic here: https://github.com/awakecoding/wireshark-rdp

EDIT: I had to put a space after '@' in the sample usernames as they were detected as email addresses and automatically removed for privacy reasons. Just assume there is no space.