Hello, Chris here from Directory Services support team with part 3 of the series.
With the November 2022 security update, some things were changed as to how the Kerberos Key Distribution Center (KDC) Service on the Domain Controller determines what encryption types are supported by the KDC and what encryption types are supported by default for users, computers, Group Managed Service Accounts (gMSA), and trust objects within the domain.
It is strongly recommended that you read the following article before going forward if you are not certain about Kerberos Encryption types are nor what is supported by the Windows Operating System:
Understanding Kerberos encryption types: https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/decrypting-the-selection-of-supported-kerberos-encryption-types/ba-p/1628797
Before we dive into what all has changed, note that there were some unexpected behaviors with the November update:
November out-of-band announcement: https://techcommunity.microsoft.com/t5/ask-the-directory-services-team/november-2022-out-of-band-update-released-take-action/ba-p/3680144
Kerberos changes related to Encryption Type: https://support.microsoft.com/en-us/topic/kb5021131-how-to-manage-the-kerberos-protocol-changes-related-to-cve-2022-37966-fd837ac3-cdec-4e76-a6ec-86e67501407d
November out-of-band guidance: https://learn.microsoft.com/en-us/windows/release-health/windows-message-center#2961
After installing Windows Updates released on November 8, 2022 on Windows domain controllers, you might have issues with Kerberos authentication.
This specific failure is identified by the logging of Microsoft-Windows-Kerberos-Key-Distribution-Center Event ID 14 in the System event log of DC role computers with this unique signature in the event message text:
While processing an AS request for target service <service>, the account <account name> did not have a suitable key for generating a Kerberos ticket (the missing key has an ID of 1). The requested etypes : <etype numbers >. The accounts available etypes : <etype numbers>. Changing or resetting the password of <account name> will generate a proper key.
Where (a.) “the missing key has an ID 1” and (b.) "4" is not listed in the "requested etypes" or "account available etypes" fields.
You need to read the links above. If you are experiencing this signature above, Microsoft strongly recommends installing the November out of band patch (OOB) which mitigated this regression. The OOB should be installed on top of or in-place of the Nov 8 update on DC Role computers while paying attention to special install requirements for Windows Updates on pre-WS 2016 DCs running on the Monthly Rollup (MR) or SO (Security only) servicing branches.
Note that this out-of-band patch will not fix all issues. You should keep reading. If you tried to disable RC4 in your environment, you especially need to keep reading.
There was a change made to how the Kerberos Key Distribution Center (KDC) Service determines what encryption types are supported and what should be chosen when a user requests a TGT or Service Ticket.
Prior to the November 2022 update, the KDC made some assumptions:
- If the User’s/GMSA’s/Computer’s/Service account’s/Trust object’s msDS-SupportedEncryptionTypes attribute was NULL (blank) or a value of 0, the KDC assumes account only supports RC4_HMAC_MD5.
- If the Windows Kerberos Client on workstations/Member Servers and KDCs are configured to ONLY support either one or both versions of AES encryption, the KDC would create an RC4_HMAC_MD5 encryption key as well as create AES Keys for the account if msDS-SupportedEncryptionTypes was NULL or a value of 0. This meant you could still get AES tickets.
- Configurations where FAST/Windows Claims/Compound Identity/Disabled Resource SID Compression were implemented had no impact on the KDC’s decision for determining Kerberos Encryption Type. See below screen shot of an example of a user account that has these higher values configured but DOES NOT have an encryption type defined within the attribute.
After November 2022 Update the KDC Makes the following decisions:
- If the User’s/GMSA’s/Computer’s/Service account’s/Trust object’s msDS-SupportedEncryptionTypes attribute was NULL (blank) or a value of 0, it defaults to an RC4_HMAC_MD5 encrypted ticket with AES256_CTS_HMAC_SHA1_96 session keys if the DefaultDomainSupportedEncTypes registry value is not configured on the KDC. If this value IS configured, then it uses the value defined in the registry.
- If the User’s/GMSA’s/Computer’s/Service account’s/Trust object’s msDS-SupportedEncryptionTypes attribute is NOT NULL nor a value of 0, it will use the most secure intersecting (common) encryption type specified. If the KDC’s Kerberos client is NOT configured to support any of the encryption types configured in the account’s msDS-SupportedEncryptionTypes attribute then the KDC will NOT issue a TGT or Service Ticket as there is no common Encryption type between the Kerberos Client, Kerberos enabled service, or the KDC.
As explained above, the KDC is no longer proactively adding AES support for Kerberos tickets, and if it is NOT configured on the objects then it will more than likely fail if RC4_HMAC_MD5 has been disabled within the environment. This literally means that the authentication interactions that worked before the 11b update that shouldn't have, correctly fail now.
On top of that, if FAST, Compound Identity, Windows Claims, or Resource SID Compression has been enabled on accounts that don’t have specific encryption types specified within the environment, it also will cause the KDC to NOT issue Kerberos tickets as the attribute msDS-SupportedEncryptionTypes is no longer NULL or a value of 0. These technologies/functionalities are outside the scope of this article. You can read more about these higher bits here: FAST, Claims, Compound auth and Resource SID compression.
Steps to address the issues:
So now that you have the background as to what has changed, we need to determine a few things.
If the November 2022/OOB updates have been deployed to your domain controller(s), determine if you are having problems with the inability for the domain controllers (KDC) to issue Kerberos TGT’s or Service tickets. This can be done by Filtering the System Event log on the domain controllers for the following:
Event Log: System
Event Source: Kerberos-Key-Distribution-Center
Event IDs: 16,27,26,14,42
NOTE: If you want to know about the detailed description, and what it means, see the section later in this article labeled: Kerberos Key Distribution Center Event error messages.
If any of these have started around the same time as the November security update being installed, then we already know that the KDC is having issues issuing TGT or Service tickets.
First, we need to determine if your environment was configured for Kerberos FAST, Compound Identity, Windows Claims or Resource SID Compression. This can be easily done one of two ways:
- Running the following Windows PowerShell command to show you the list of objects in the domain that are configured for these.
Get-ADObject -Filter "msDS-supportedEncryptionTypes -bor 0xf0000 -and -not msDS-supportedEncryptionTypes -bor 0x1c"
- Running the 11B checker (see sample script
at the end of the articlefound at GitHub - takondo/11Bchecker ) and checking the list of accounts under the section, “There are X objects that have msDS-SupportedEncryptionTypes configured, but no encryption protocol is allowed.”
If any objects are returned, then the supported encryption types will be REQUIRED to be configured on the object’s msDS-SupportedEncryptionTypes attribute. If the script returns a large number of objects in the Active Directory domain, then it would be best to add the encryption types needed via another Windows PowerShell command below:
Set-ADUser [sAMAccountName] -KerberosEncryptionType [CommaSeparatedListOfEtypes]
Set-ADComputer [sAMAccountName] -KerberosEncryptionType [CommaSeparatedListOfEtypes]
Set-ADServiceAccount [sAMAccountName] -KerberosEncryptionType [CommaSeparatedListOfEtypes]
Supported values for ETypes: DES, RC4, AES128, AES256
NOTE: The value “None” is also supported by the PowerShell Cmdlet, but will clear out any of the supported encryption types.
If no objects are returned via method 1, or 11B checker doesn’t return any results for this specific scenario, it would be easier to modify the default supported encryption type for the domain via a registry value change on all the domain controllers (KDCs) within the domain. This is done by adding the following registry value on all domain controllers.
HKLM\System\CurrentControlSet\Services\KDC
Value Type: REG_DWORD
Value Name: DefaultDomainSupportedEncTypes
The value data required would depend on what encryption types that are required to be configured for the domain or forest for Kerberos Authentication to succeed again.
Some of the common values to implement are:
For AES128_CTS_HMAC_SHA1_96 and AES256_CTS_HMAC_SHA1_96 support, you would set the value to: 0x18. This will exclude use of RC4 on accounts with msDS-SupportedEncryptionTypes value of NULL or 0 and require AES.
For RC4_HMAC_MD5, AES128_CTS_HMAC_SHA1_96 and AES256_CTS_HMAC_SHA1_96 support, you would set the value to: 0x1C. This will allow use of both RC4 and AES on accounts when msDS-SupportedEncryptionTypes value of NULL or 0. Note: This will allow the use of RC4 session keys, which are considered vulnerable.
If you want to include an AES256_CTS_HMAC_SHA1_96_SK (Session Key), then you would add 0x20 to the value. So, if with the above examples 0x18 would be 0x38, and 1C would be 0x3C.
More information about the supported encryption type bit flags can be found here:
The KDC registry value can be added manually on each domain controller, or it could be easily deployed throughout the environment via Group Policy Preference Registry Item deployment.
Mismatched Kerberos Encryption Types
The next issue needing attention is the problem of mismatched Kerberos Encryption Types and missing AES keys. You can leverage the same 11b checker script mentioned above to look for most of these problems.
Here’s an example of an environment that is going to have problems with explanations in the output (Note: This script does not make any changes to the environment. It just outputs a report to the screen):
*****************************************
Legacy OS detected: CN=F42003,CN=Computers,DC=contoso,DC=com
This OS is not compatible with the new default behavior, and authentication to this computer will fail after the DC is upgraded to 11B/11OOB
Explanation: This computer is running an unsupported Operating System that requires RC4 to be enabled on the domain controller.
There are 2 objects that do not have msDS-SupportedEncryptionTypes configured.
When authenticating to this target, Kerberos will default to the setting of DefaultDomainSupportedEncTypes registry on the authenticating DC.
This defaults to a value of 0x27, which means 'use AES for session keys and RC4 for ticket encryption'
If this target service does not support AES, please set msDS-SupportedEncryptionTypes to 4 on this object so that only RC4 is used.
CN=alice,CN=Users,DC=contoso,DC=com
CN=bob,CN=Users,DC=contoso,DC=com
Explanation: If you have disabled RC4, you need to manually set these accounts accordingly, or leverage DefaultDomainSupportedEncTypes. If you still have RC4 enabled throughout the environment, no action is needed.
======================================
There are 1 objects that have msDS-SupportedEncryptionTypes configured, but no encryption protocol is allowed.
This can cause authentication to/from this object to fail.
Please either delete the existing msDS-SupportedEncryptionTypes settings, or add supported etypes.
Example: Add 0x1C to signify support for AES128, AES256, and RC4
CN=server12,CN=Users,DC=contoso,DC=com
Explanation: The fix action for this was covered above in the FAST/Windows Claims/Compound Identity/Resource SID compression section.
======================================
There are 2 computers/services that are configured for RC4/DES only
If you have any clients or DCs that are configured to only support AES, authentication will fail
Here is the list of objects that are RC4/DES only:
CN=computer4,CN=Users,DC=contoso,DC=com
CN=Stefan,CN=Users,DC=contoso,DC=com
Explanation: If are trying to enforce AES anywhere in your environments, these accounts may cause problems. You need to investigate why they have been configured this way and either reconfigure, update, or replace them.
A common scenario where authentication fails after installing November update on DCs in this condition is if DCs are configured to only support AES
Example: Setting the 'Configure encryption types allowed for Kerberos' policy to AES only on DCs
Here are the DCs configured for AES only:
CN=DC1,OU=Domain Controllers,DC=contoso,DC=com
Explanation: This is warning you that RC4 is disabled on at least some DCs. You’ll need to consider your environment to determine if this will be a problem or is expected.
Want more Information about Windows OS and Kerberos?
Attribute msDS-SupportedEncryptionTypes
To avoid redundancy, I will briefly cover a very important attribute called msDS-SupportedEncryptionTypes on objectClasses of User. For our purposes today, that means user, computer, and trustedDomain objects. Here's an example of that attribute on a user object:
If you haven’t patched yet, you should still check for some issues in your environment prior to patching via the same script mentioned above.
If you have already patched, you need to keep an eye out for the following Kerberos Key Distribution Center events. If you see any of these, you have a problem.
Event ID 14 Description: While processing an AS request for target service krbtgt/contoso.com, the account Client$ did not have a suitable key for generating a Kerberos ticket (the missing key has an ID of 5). The requested etypes : 18 17 23 3 1. The accounts available etypes : 23. Changing or resetting the password of krbtgt will generate a proper key.
Translation: The krbtgt account has not been reset since AES was introduced into the environment.
Resolution: Reset the krbtgt account password after ensuring that AES has not been explicitly disabled on the DC.
Event ID 42 Description: The Kerberos Key Distribution Center lacks strong keys for account krbtgt. You must update the password of this account to prevent use of insecure cryptography.
Translation: The krbtgt account has not been reset since AES was introduced into the environment.
Resolution: Reset the krbtgt account password after ensuring that AES has not been explicitly disabled on the DC.
Event ID 26 Description: While processing an AS request for target service krbtgt/CONTOSO.COM, the account Client$ did not have a suitable key for generating a Kerberos ticket (the missing key has an ID of 3). The requested etypes were 18. The accounts available etypes were 23 18 17.
Translation: The DC, krbtgt account, and client have a Kerberos Encryption Type mismatch.
Resolution: Analyze the DC and client to determine why the mismatch is occurring.
Event ID 16 Description: While processing a TGS request for the target server http/foo.contoso.com, the account admin@contoso.com did not have a suitable key for generating a Kerberos ticket (the missing key has an ID of 8). The requested etypes were 18 17 23 24 -135. The accounts available etypes were 23 18 17. Changing or resetting the password of <account name> will generate a proper key.
Translation: The encryption types specified by the client do not match the available keys on the account or the account’s encryption type configuration.
Resolution: Reset <account name’s>password after ensuring that AES has not been explicitly disabled on the DC or ensure that the client’s and service account’s encryption types have a common algorithm.
Event ID 27 Description: While processing a TGS request for the target server http/foo.contoso.com, the account admin@CONTOSO.COM did not have a suitable key for generating a Kerberos ticket (the missing key has an ID of 9). The requested etypes were 23 3 1. The accounts available etypes were 23 18 17.
Translation: The encryption types configured on the service account for foo.contoso.com are not compatible with the encryption types specific by the DC. (Another Kerberos Encryption Type mismatch)
Resolution: Analyze the DC, the service account that owns the SPN, and the client to determine why the mismatch is occurring.
All of the events above would appear on DCs. There is one more event I want to touch on, but would be hard to track since it is located on the clients in the System event log.
Event log: System
Source: Security-Kerberos
Event ID: 4
Description: The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server ADATUMWEB$. The target name used was HTTP/adatumweb.adatum.com. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Ensure that the target SPN is only registered on the account used by the server. This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service. Ensure that the service on the server and the KDC are both configured to use the same password. If the server name is not fully qualified, and the target domain (ADATUM.COM) is different from the client domain (CONTOSO.COM), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.
Possible problem: Account hasn't had its password reset (twice) since AES was introduced to the environment or some encryption type mismatch.
Translation: There is a mismatch between what the requesting client supports and the target service account.
Resolution: Analyze the service account that owns the SPN and the client to determine why the mismatch is occurring.
By now you should have noticed a pattern. Things break down if you haven’t reset passwords in years, or if you have mismatched Kerberos Encryption policies. Keep in mind the following rules/items:
- If you have still pre Windows 2008/Vista Servers/Clients:
Such devices may still obtain Kerberos tickets, but cannot decrypt tickets with AES session keys generated by November patched DCs. Pre 2008 devices do not support the msDS-SupportedEncryptionTypes attribute. Upgrade these server to a supported version of Windows. IT Admins and CTOs should reread the Sarbanes–Oxley Act of 2002. -
If you have other third-party Kerberos clients (Java, Linux, etc.) systems that are currently using RC4 or DES:
Contact the third-party vendor to see if the device/application can be reconfigured or updated to support AES encryption, otherwise replace them with devices/applications that support AES encryption and AES session keys.
To run a command on Linux to dump the supported encryption types for a keytab file:klist -kte /var/kerberos/krb5/user/KeyTabFileName
To analyze a KeyTab file on Windows, you can use the following command to find out the encryption types configured in the file:KTPass /in KeyTabFileName
- An entire forest and all trusts should have a common Kerberos encryption type to avoid a likely outage. You must ensure that msDS-SupportedEncryptionTypes are also configured appropriately for the configuration you have deployed
- If your security team gives you a baseline image or a GPO that has RC4 disabled, and you haven’t finished prepping the entire environment to solely support AES, point them to this article. Make sure they accept responsibility for the ensuing outage.
- You'll want to leverage the security logs on the DC throughout any AES transition effort looking for RC4 tickets being issued. You need to enable auditing for "Kerberos Authentication Service" and "Kerberos Service Ticket Operations" on all Domain Controllers. Events 4768 and 4769 will be logged that show the encryption type used. The field you'll need to focus on is called "Ticket Encryption Type" and you're looking for 0x17. 0x17 indicates RC4 was issued. This XML query below can be used to filter for these:
<QueryList>
<Query Id="0" Path="Security">
<Select Path="Security">*[EventData[Data[@Name='TicketEncryptionType'] and (Data='0x17')]]
</Select>
</Query>
</QueryList>
- You need to evaluate the passwordLastSet attribute for all user accounts (including service accounts) and make sure it is a date later than when Windows Server 2008 (or later) DCs were introduced into the environment.
- Previous guidance said “Do not wait for AES to be enforced on you.” This guidance stands.
Happy Hunting,
Chris Cartwright
Sample Script
Update 12/17/2022:
The sample script "11B checker" text previously found at the bottom of this post has been removed. The script is now available for download from GitHub at GitHub - takondo/11Bchecker.
It includes enhancements and corrections since this blog post's original publication. If you obtained a version previously, please download the new version.
Related blogs:
Introduction to this blog series: https://techcommunity.microsoft.com/t5/ask-the-directory-services-team/having-issues-since-deploying...
Part 2 of this blog series: https://techcommunity.microsoft.com/t5/ask-the-directory-services-team/so-you-say-your-dc-s-memory-is-getting-all-used-up-after/ba-p/3696318
Microsoft
