I’m not a fan of folks that glaze over problems, not that I’m a pessimist mind you, I do my fair share of finding the bright side of bad situations. When this RC4 project started, I heard lots of optimistic bluster: “we’ll just do xxxx”… “all it will take is yyyy” and it quickly switched from a “recon and report” project to a “go do” effort in the blink of an eye. Over a year into the fight and I can tell you that no one could estimate how hard it would be to rid an environment of the silent killer. Yep, I called it that. Mostly because early efforts at treated it as some sort of a colorless/odorless gas that no one could see or detect. I’m here to tell you today that unless you can put facts and data around this problem, you don’t have a chance at eliminating it. Context and proof are the keys…and perseverance…lots and lots of perseverance. So how do you put context and proof around this? Well, my pain is your gain, and this post will share some of the truly effective nuggets from a real life RC4 elimination project that I hope will help you in your endeavors to rid your environment of this security problem.
As you may have gleamed from the title, this is part 3 of the series that I’ve written to date on this topic. The first two were really the early stages where surprises were encountered when security settings started coming into the environments removing RC4. The article links for the previous posts are below for your reading enjoyment.
The RC4 Removal Files Part 1: What's in an error message? - Microsoft Tech Community
The RC4 Removal Files Part 2: In AES We Trust - Microsoft Tech Community
Knowledge is power
The best source of truth for monitoring systems that are still using RC4 are the event logs from the DCs. Identifying machines/accounts that have problems is the early detection system that provided all telemetry for this effort. There are some events that give direct information about the negotiation of Kerberos tickets and there are some events that were used as sort of an indicator that there might be problems. The first pre-requisite for this hunt is to enable additional Kerberos logging on the domain controllers. The link below is a great resource to complete this task:
Enable Kerberos event logging - Windows Server | Microsoft Docs
The defaults on most Windows systems are set to be pliable enough to work in a secure environment but tolerate a lower security environment. Now, if you work in a tightly regulated environment and settings are more tightly controlled, modern Windows systems that are secured can talk to other modern Windows that are secured. But this is where it gets interesting and difficult. Environments contain many non-windows platforms that have to be configured, or in some cases updated, to support changes to protocols or authentication that come along after the devices are deployed. How do you go about identifying them? Well let’s get into that.
Below are the events that are directly involved in the hunt to identify RC4 as well as some other settings to check when you locate machines or devices that are not configured to use more modern encryption settings. For each event, the PowerShell commands to query the domain controller’s event logs are included below. Getting the events on a larger scale is also covered later so keep reading.
Security Event 4769 with 0x17 (RC4) negotiation:
|
Get-WinEvent -FilterXPath “*[System[EventID=4769] and EventData[Data[@Name=’TicketEncryptionType’]=’0x17’]]” -LogName Security |
Note: Command above is a single line
This event will identify the encryption type negotiated during a Kerberos ticket transaction and requires additional logging to be enabled on the domain controllers. It’s only effective if the domain controllers are still configured to accept RC4, otherwise there should be no 0x17 (RC4) tickets negotiated.
4769(S, F) A Kerberos service ticket was requested. (Windows 10) - Windows security | Microsoft Docs
Directory Services Event 2889 rejecting unsigned or simple LDAP binds:
| Get-WinEvent -FilterHashTable @{LogName=’Directory Service’;Id=’2889’;StartTime=((Get-date).AddDays(-7))} |
Note: Command above is a single line
If you have your domain controllers configured to require LDAP signing, and let’s hope you do, the above-mentioned events will show devices that are attempting to make either unsigned binds or simple binds. They are both insecure situations but of the two, the simple binds are the most dangerous since the sending end of the LDAP bind is putting the password in plain text on the network. The sending end is stressed in that sentence because the domain controllers are rejecting the connection, but nothing is stopping the sender from attempting the connection. By the time the event is logged, the password is already out there. Hopefully you see the reason to track those devices down and fix them and reset their passwords once remediated.
Unsigned binds are susceptible to man-in-the middle attacks, so it is critical to ensure that the client LDAP settings are set to require signing for Windows systems. For 3rd-party systems/devices, refer to vendor documentation for configuration of secure LDAP binds. The article below goes into more detail if needed:
How to enable LDAP signing - Windows Server | Microsoft Docs
System Event 5827 rejecting unsigned netlogon connections
|
Get-WinEvent -FilterHashTable @{LogName=’System’;Id=’5827’;StartTime=((Get-date).AddDays(-7))} |
Note: Command above is a single line
This event is the NetLogon equivalent of the LDAP signing event mentioned above and was released in a phased deployment between August 2020 and February 2021. The February 2021 part of this effort forced domain controllers into the Enforcement Phase, meaning the domain controllers will require NetLogon signing regardless of the configured registry settings that were part of the initial rollout of this setting. The link below contains more details if needed. It is important to note that Windows devices should be configured by default to perform the signed NetLogon connections. 3rd-party devices might need configuration changes or firmware/software updates to support this functionality so, again work with the vendor to apply these settings or updates.
At this point you may be wondering what the last two examples have to do with RC4…the answer is not much (hence the “Everything Else” in the article title). They do however relate to machines that are not keeping current with vulnerability patching or best-practice for security configuration. Chances are high that you are going to see some overlap between the machines that cannot make a signed LDAP or Netlogon connection and the machines negotiating RC4. We might as well fix all problems while we’re at it. Specifically, requirements for signed Netlogon connections (CVE-2020-1472) and LDAP Channel Binding are recent settings that should be monitored for compliance.
Tricks of the trade:
Below are the items that are particularly effective in identifying problems and gathering evidence in the effort.
Being able to prove what’s going on in Kerberos negotiation is priceless in the data collection/recon phase as well as during the remediation phase. As with most efforts being able to filter out the noise in a network capture is key so something along the lines of the filter below will get you started:
Network capture filter:
|
(ip.address == X.X.X.X or ip.address == Y.Y.Y.Y) and (kerberos) |
Kerberos command line tools:
klist & klist get host/yourserverhere to see what is being negotiated in the Kerberos tickets
- KList with no parameters will show the negotiated encryption type after accessing a resource.
- Klist GET host/ServerName requests a ticket against a specific SPN on a server and can be used to force negotiation of a key for troubleshooting.
Good References:
Replace the word “DES” with “RC4” in Ned Pyle’s article, it’s an oldie but a goodie and still applies.
Hunting down DES in order to securely deploy Kerberos - Microsoft Tech Community),
Decrypting the Selection of Supported Kerberos Encryption Types by Jerry Devore is a fantastic reference to have on hand for all facets of this hunt.
Decrypting the Selection of Supported Kerberos Encryption Types - Microsoft Tech Community
[MS-KILE]: Kerberos Protocol Extensions, it’s like reading a phone book in another language for
the uninitiated but it is the definitive guide (drink coffee…it helps)
[MS-KILE]: Kerberos Protocol Extensions | Microsoft Docs
Scripting:
Grabbing and de-duplicating events from the domain controllers is a staple of this effort. If you’re lucky enough to have a small number of DCs, it might be doable to log in to all machines and gather data. Some environments have hundreds or thousands of DCs to monitor so looping through them all might be required via script. Below is a very simple loop to go through all DCs in the domain which might be a good starting point for smaller environments. For extremely large environments, blending this starter script with Jobs and Invoke-Command could be a good fit, see Parallel Processing with jobs in PowerShell - Scripting Blog (microsoft.com) for more information on this topic.
[array]$AllMyDCs = (Get-ADDomainController – Filter *).Name
ForEach ($DC in $allMyDCs) {
### LINE BELOW MAY BE FORMATTED ON PAGE INCORRECTLY, IT IS ONE LINE ####
[array]$evts = Get-WinEvent -Computername $DC -FilterXPath “*[System[EventID=4769] and EventData[Data[@Name=’TicketEncryptionType’]=’0x17’]]” -LogName Security
[array]$evtsOut = @{}
ForEach ($evt in $evts) {
$evtXML = ([xml]$Evt.toXML()).event
$evtData = $evtXML.EventData.Data.’#text’
$trgtUsr = $evtData[0]
$tcktOpts = $evtData[4]
$tcktEncr = $evtData[5]
$IPAddr = ($evtData[6]).replace(“::ffff:”,””)
$status = $evtData[8]
$svcNm = $evtData[2]
$objRec = [psCustomObject] @{
TargetUser = $trgtUsr
TicketOptions = $tcktOpts
TicketEncryption = $tcktEncr
IPAddress = $IPAddr
Status = $status
ServiceName = $svcNm
}
If ($evtsOut.Count -gt 0) {
If ($evtsOut -notMatch $trgtusr) {
$evtsOut += $objRec
$evtsOut = $evtsOut | Sort-Object -Property TargetUser -unique
}
}
Else {
$evtsOut += $objRec
}
}
}
Event forwarding/collection:
To the point above it may make sense to consolidate events to a smaller group of event collectors which greatly simplifies scripting this. If you have the capability to forward events to a WEC to simplify the locations you have to query, by all means leverage that capability.
The Human Factor
As the system remediation evolves, you are likely to get a wide array of reactions that I hope I can help you head off. Suffice it to say, "It's always been this way" is the enemy here but I'm sharing some of the tactics that I found unique in the process of bringing security into check.
“My stuff can’t be broke” or “It’s Not Me, It’s You”
When you start monitoring the event logs, its important to look at the machine account in the connections. There may be surprising connections being made by network appliances, multi-function printer/scanners, or storage devices (SAN/NAS), just to name a few. Since these items typically fall outside of the purview of GPOs or even patch compliance tracking, they may lag in getting set to the most secure settings.
My advice here is to not let the tail wag the dog. These device owners may try to make a case that these settings broke their systems. That conversation needs to be flipped around to discuss how accommodating that device affects the security posture of the forest.
Your account is so old…
If smart card authentication is in use, make sure that you verify old accounts. A good rule of thumb is to look at the passwordLastSet attribute and the whenCreated/whenChanged attributes for accounts that show up on the RC4 lists. You might even find some accounts that have never had the password set, or in the case of Computer accounts might have been joined to the domain so far back that they need an adjustment to get with the times. There are various possibilities for this scenario one if which could be migration from other domains/forests. For these edge cases, the account settings for msds-supportedEncryptionTypes can be set to ensure that the account in question will be forced to use AES.
That’s All Folks (…I Hope!)
We live in an exciting time. The world appears to be on the cusp of some magnificent breakthroughs in computing power that will render everything regarding encryption in this article look like simple addition. It’s imperative that you get current and stay current. As a former college instructor told us back in the early 90’s, “If you want to go into something that doesn’t change, switch your major to History now, otherwise buckle up and enjoy the ride”.
Thanks for reading!
Disclaimer
The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.