Symptom
Sometime in your SF clusters, you may observe that nodes being down with “unable to read private key from certificate” and checking the Service Fabric Admin logs, you will see error like below:
SecurityUtility
Failed to get the Certificate's private key. Thumbprint:XXXXXXXXXXXXXXXXX…..XXXXXXXX. Error: E_FAIL
CryptoUtility
CryptAcquireCertificatePrivateKey failed. Error: 0x80090014
Error code: 0x80090014 meaning “Invalid provider type specified.”
Mitigation
Hence to confirm if you’re hitting into the issue where the certificate can’t be ACLed by the SF runtime due to the fact that the certificate being generated with an unsupported provider, then please try the following command in PowerShell by logging into the node, from which the error is thrown.
cd Cert:\LocalMachine\My
certutil -store my
This will dump all the certificates with its details from ‘my’ store, you can now look for your concerned certificate with the help of the thumbprint and check what provider the certificate has:
If you see the provider of the certificate contains something like below, then this is indeed a CNG certificate issued with a Key Storage Provider.
Provider = Microsoft Software Key Storage Provider
As of now SF runtime supports certificates with providers as mentioned here - https://docs.microsoft.com/en-us/windows/win32/seccrypto/microsoft-cryptographic-service-providers
Hence, you might be using a self-signed certificate which was generated without any providers specified, had used a CNG provider instead. If this is the case, then you may need to create another certificate with a supported provider that you can associate with this cluster using following command:
New-SelfSignedCertificate -NotBefore '<Values>' -NotAfter '<Values>' -DnsName '<DnsName>' -CertStoreLocation Cert:\LocalMachine\My -Provider "Microsoft Enhanced RSA and AES Cryptographic Provider" -KeyExportPolicy ExportableEncrypted -Subject "<Enter Subject>"
After creating this certificate, you can add this new certificate as the secondary certificate to the cluster and then swap this with the primary to avoid any down time - https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-cluster-security-update-certs-azure.
Updated Sep 15, 2020
Version 4.0Braja
Microsoft
Joined July 23, 2019
Azure PaaS Blog
Follow this blog board to get notified when there's new activity