Forum Discussion
did the Jun 9 Security Update remove trusted (amazon) certs?
We have a product that runs on Windows and uses AWS IoT to connect and transmit info.
We noticed yesterday that many - roughly 2/3 of our fleet went silent - no connections.
We have some of our own PCs that experienced this as well. Investigation yielded issues with TLS. The client (which is just using the default windows cert checking) actively terminated the connected. It didn't like the cert from the AWS IoT endpoint.
All of these PCs were happily connected on Jun 8.
Investigation seemed to indicate that some combination of Amazon Root CA (1-4) and some Starfield certs were not in the device cert mgr.
Also - when we manually added AmazonRootCA1.pem to the cert mgr, our service connected again.
So the evidence seems to strongly support that the security update removed trusted amazon root certs from the cert store.
I'm guessing some/many won't notice since they are making regular TCP connections and maybe the certs get auto-added if they are not there? But we are doing MQTT over the AWS port 8883. So perhaps Windows did not detect this and seek to refresh its cert store?
Can anyone confirm if they have seen the same?
1 Reply
- Hellen-CharlessIron Contributor
We haven't seen it across our environment, but your findings do sound consistent with a trust store issue rather than an AWS IoT problem.
If manually importing Amazon Root CA 1 immediately restores the connection, that strongly suggests the TLS handshake is failing because the client can no longer build a trusted certificate chain. AWS IoT endpoints themselves haven't had a broad certificate change that would explain this behavior on its own.
One thing I would check is whether the affected machines have Automatic Root Certificates Update disabled (via Group Policy or registry) or if they were unable to reach Microsoft's root certificate update service. Windows doesn't necessarily have every trusted root preinstalled it can download roots on demand under the right conditions.
It's also interesting that only about two-thirds of your fleet was affected. That makes me wonder if there's a common factor, such as:
Different Windows builds or patch levels
Different GPOs affecting certificate management
Machines that haven't accessed HTTPS endpoints using the Amazon trust chain recently
Restricted internet access preventing automatic root updates
I'd be interested to know which Windows versions and KB updates the affected systems have installed. If they're all on the same cumulative update, that could help determine whether this was caused by a Windows update or by differences in how the root certificate store was maintained.