Not able to use PFX format certificate in SQL Server?

Published Jan 15 2019 04:38 PM 446 Views
Microsoft
First published on MSDN on Jun 18, 2015

[Update  8/24/2015


This issue has been fixed by SQL Server 2014 SP1 cumulative update 2.  See KB https://support.microsoft.com/en-us/kb/3082513 for more details.


]



If you have a security certificate in PFX format that is generated by Microsoft Certificate store, you can not use it directly in SQL Server.   But you can use PVKConverter.exe tool to convert to PVK/DER format that can be used by SQL Server.   KB “ How to use PFX-formatted certificates in SQL Server ” has documentation on using this tool.


We have a customer who reported to us that they were not able to use their certificate even after they did the conversion.  They got various errors like below:



Msg 15297, Level 16, State 56, Line 1 The certificate, asymmetric key, or private key data is invalid.


Msg 15474, Level 16, State 6, Line 8 Invalid private key. The private key does not match the public key of the certificate.



After digging and debugging, we learned that  it is because the serial number of their certificate was too long.   Currently SQL Server only allows serial number up to 16 bytes.   But customer’s certificate had 19 bytes for the serial number.


You can check your certificate’s serial number by using certutil.exe –dump option or just use certificate manager (certmgr.msc) and check the property details as shown below.  In this example, the serial number is exactly 16 bytes.





Now the question is why customer’s certificate had 19 bytes of serial number?  They told me that they generated the certificate using Microsoft Certificate store.


It turned out that you can actually have some control over the serial number through HighSerial as documented in “ Custom CA Configuration ”.   If you set it to 0 like ( “certutil -setreg ca\highserial 0” ), you will get 10 byte serial number for future certificate generation (after you configure and restart your certificate service).   There are various other options in the document that you can explore and control length and content of your certificate’s serial number.



Update


This issue has been fixed by SQL Server 2014 SP1 cumulative update 2.  See KB https://support.microsoft.com/en-us/kb/3082513 for more details.




Jack Li |Senior Escalation Engineer | Microsoft SQL Server


twitter | pssdiag | Sql Nexus

%3CLINGO-SUB%20id%3D%22lingo-sub-318593%22%20slang%3D%22en-US%22%3ENot%20able%20to%20use%20PFX%20format%20certificate%20in%20SQL%20Server%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-318593%22%20slang%3D%22en-US%22%3E%0A%20%26lt%3Bmeta%20http-equiv%3D%22Content-Type%22%20content%3D%22text%2Fhtml%3B%20charset%3DUTF-8%22%20%2F%26gt%3B%3CSTRONG%3E%20First%20published%20on%20MSDN%20on%20Jun%2018%2C%202015%20%3C%2FSTRONG%3E%20%3CBR%20%2F%3E%3CP%3E%3CSTRONG%3E%20%5BUpdate%26nbsp%3B%208%2F24%2F2015%20%3C%2FSTRONG%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3EThis%20issue%20has%20been%20fixed%20by%20SQL%20Server%202014%20SP1%20cumulative%20update%202.%26nbsp%3B%20See%20KB%20%3CA%20href%3D%22https%3A%2F%2Fsupport.microsoft.com%2Fen-us%2Fkb%2F3082513%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3E%20https%3A%2F%2Fsupport.microsoft.com%2Fen-us%2Fkb%2F3082513%20%3C%2FA%3E%20for%20more%20details.%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%5D%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3EIf%20you%20have%20a%20security%20certificate%20in%20PFX%20format%20that%20is%20generated%20by%20Microsoft%20Certificate%20store%2C%20you%20can%20not%20use%20it%20directly%20in%20SQL%20Server.%26nbsp%3B%26nbsp%3B%20But%20you%20can%20use%20%3CA%20href%3D%22http%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fdownload%2Fdetails.aspx%3Fid%3D40812%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3E%20PVKConverter.exe%20%3C%2FA%3E%20tool%20to%20convert%20to%20PVK%2FDER%20format%20that%20can%20be%20used%20by%20SQL%20Server.%26nbsp%3B%26nbsp%3B%20KB%20%E2%80%9C%20%3CA%20href%3D%22https%3A%2F%2Fsupport.microsoft.com%2Fen-us%2Fkb%2F2914662%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3E%20How%20to%20use%20PFX-formatted%20certificates%20in%20SQL%20Server%20%3C%2FA%3E%20%E2%80%9D%20has%20documentation%20on%20using%20this%20tool.%3C%2FP%3E%3CBR%20%2F%3E%3CP%3EWe%20have%20a%20customer%20who%20reported%20to%20us%20that%20they%20were%20not%20able%20to%20use%20their%20certificate%20even%20after%20they%20did%20the%20conversion.%26nbsp%3B%20They%20got%20various%20errors%20like%20below%3A%3C%2FP%3E%3CBR%20%2F%3E%3CBLOCKQUOTE%3E%3CBR%20%2F%3E%3CP%3EMsg%2015297%2C%20Level%2016%2C%20State%2056%2C%20Line%201%20The%20certificate%2C%20asymmetric%20key%2C%20or%20private%20key%20data%20is%20invalid.%3C%2FP%3E%3CBR%20%2F%3E%3CP%3EMsg%2015474%2C%20Level%2016%2C%20State%206%2C%20Line%208%20Invalid%20private%20key.%20The%20private%20key%20does%20not%20match%20the%20public%20key%20of%20the%20certificate.%3C%2FP%3E%3CBR%20%2F%3E%3C%2FBLOCKQUOTE%3E%3CBR%20%2F%3E%3CP%3EAfter%20digging%20and%20debugging%2C%20we%20learned%20that%26nbsp%3B%20it%20is%20because%20the%20serial%20number%20of%20their%20certificate%20was%20too%20long.%26nbsp%3B%26nbsp%3B%20Currently%20SQL%20Server%20only%20allows%20serial%20number%20up%20to%2016%20bytes.%26nbsp%3B%26nbsp%3B%20But%20customer%E2%80%99s%20certificate%20had%2019%20bytes%20for%20the%20serial%20number.%3C%2FP%3E%3CBR%20%2F%3E%3CP%3EYou%20can%20check%20your%20certificate%E2%80%99s%20serial%20number%20by%20using%20certutil.exe%20%E2%80%93dump%20option%20or%20just%20use%20certificate%20manager%20(certmgr.msc)%20and%20check%20the%20property%20details%20as%20shown%20below.%26nbsp%3B%20In%20this%20example%2C%20the%20serial%20number%20is%20exactly%2016%20bytes.%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F68328iF62B62A21742A46C%22%20%2F%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3ENow%20the%20question%20is%20why%20customer%E2%80%99s%20certificate%20had%2019%20bytes%20of%20serial%20number%3F%26nbsp%3B%20They%20told%20me%20that%20they%20generated%20the%20certificate%20using%20Microsoft%20Certificate%20store.%3C%2FP%3E%3CBR%20%2F%3E%3CP%3EIt%20turned%20out%20that%20you%20can%20actually%20have%20some%20control%20over%20the%20serial%20number%20through%20HighSerial%20as%20documented%20in%20%E2%80%9C%20%3CA%20href%3D%22https%3A%2F%2Ftechnet.microsoft.com%2Fen-us%2Flibrary%2Fcc784789(v%3Dws.10).aspx%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3E%20Custom%20CA%20Configuration%20%3C%2FA%3E%20%E2%80%9D.%26nbsp%3B%26nbsp%3B%20If%20you%20set%20it%20to%200%20like%20(%20%E2%80%9Ccertutil%20-setreg%20ca%5Chighserial%200%E2%80%9D%20)%2C%20you%20will%20get%2010%20byte%20serial%20number%20for%20future%20certificate%20generation%20(after%20you%20configure%20and%20restart%20your%20certificate%20service).%26nbsp%3B%26nbsp%3B%20There%20are%20various%20other%20options%20in%20the%20document%20that%20you%20can%20explore%20and%20control%20length%20and%20content%20of%20your%20certificate%E2%80%99s%20serial%20number.%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3CSTRONG%3E%20Update%20%3C%2FSTRONG%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3EThis%20issue%20has%20been%20fixed%20by%20SQL%20Server%202014%20SP1%20cumulative%20update%202.%26nbsp%3B%20See%20KB%20%3CA%20href%3D%22https%3A%2F%2Fsupport.microsoft.com%2Fen-us%2Fkb%2F3082513%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3E%20https%3A%2F%2Fsupport.microsoft.com%2Fen-us%2Fkb%2F3082513%20%3C%2FA%3E%20for%20more%20details.%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3C%2FP%3E%3CBR%20%2F%3E%3CP%3EJack%20Li%20%7CSenior%20Escalation%20Engineer%20%7C%20Microsoft%20SQL%20Server%3C%2FP%3E%3CBR%20%2F%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftwitter.com%2Fjackli8898%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3E%20twitter%20%3C%2FA%3E%20%7C%20%3CA%20href%3D%22http%3A%2F%2Fdiagmanager.codeplex.com%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3E%20pssdiag%20%3C%2FA%3E%20%7C%20%3CA%20href%3D%22http%3A%2F%2Fsqlnexus.codeplex.com%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3E%20Sql%20Nexus%20%3C%2FA%3E%3C%2FP%3E%0A%20%0A%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-318593%22%20slang%3D%22en-US%22%3EFirst%20published%20on%20MSDN%20on%20Jun%2018%2C%202015%20%5BUpdate%26nbsp%3B%208%2F24%2F2015This%20issue%20has%20been%20fixed%20by%20SQL%20Server%202014%20SP1%20cumulative%20update%202.%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-318593%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ESecurity%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Version history
Last update:
‎Jan 15 2019 04:38 PM
Updated by: