TDS/SSL issues

Copper Contributor

Hello,
I am developing a Java-based proxy front-end that sits in front of a SQL Server instance and communicates with it using the latest TDS protocol. I am facing issues with the way SQL server does SSL-over-TDS handshake in the initial phase of creating an encrypted connection: SQL Server fails to finish the SSL handshake. There is nothing in the SQL Server logs to hint to what is going on and why this is happening - I am testing against a black box. This is a fairly low level, SSL-over-TDS handshake issue - is there any way I can get developer support, most likely from Microsoft SQL server engineers?
I have full logs and traces for what is happening on the front-end side, but since there are no helpful logs in SQL Server, I will probably need someone with access to the source code running an instance on the opposite side.

2 Replies

Here is summary of what I see from the client. I also attached the  full log with more details (I can explain).

 

Client: Java 16 app running on MacOs M1 BigSur 11.5.2 (latest)

Server: running on VirtualBox VM 

    SQL Express 2019 (latest, 3-4 months  old)

    Windows Server 2019 (latest, 3-4 months old)

 

1. Client sends Hello message:
-------------------
javax.net.ssl|WARNING|69|macbook2-thread-2|2021-09-07 06:36:47.780 PDT|ServerNameExtension.java:268|Unable to indicate server name
javax.net.ssl|DEBUG|69|macbook2-thread-2|2021-09-07 06:36:47.780 PDT|SSLExtensions.java:260|Ignore, context unavailable extension: server_name
javax.net.ssl|INFO|69|macbook2-thread-2|2021-09-07 06:36:47.781 PDT|AlpnExtension.java:182|No available application protocols
javax.net.ssl|DEBUG|69|macbook2-thread-2|2021-09-07 06:36:47.781 PDT|SSLExtensions.java:260|Ignore, context unavailable extension: application_layer_protocol_negotiation
javax.net.ssl|DEBUG|69|macbook2-thread-2|2021-09-07 06:36:47.781 PDT|SessionTicketExtension.java:408|Stateless resumption supported
javax.net.ssl|DEBUG|69|macbook2-thread-2|2021-09-07 06:36:47.781 PDT|SSLExtensions.java:260|Ignore, context unavailable extension: cookie
javax.net.ssl|DEBUG|69|macbook2-thread-2|2021-09-07 06:36:47.790 PDT|SSLExtensions.java:260|Ignore, context unavailable extension: renegotiation_info
javax.net.ssl|DEBUG|69|macbook2-thread-2|2021-09-07 06:36:47.791 PDT|PreSharedKeyExtension.java:662|No session to resume.
javax.net.ssl|DEBUG|69|macbook2-thread-2|2021-09-07 06:36:47.791 PDT|SSLExtensions.java:260|Ignore, context unavailable extension: pre_shared_key
javax.net.ssl|DEBUG|69|macbook2-thread-2|2021-09-07 06:36:47.792 PDT|ClientHello.java:652|Produced ClientHello handshake message (
"ClientHello": {
"client version" : "TLSv1.2",
"random" : "51 80 77 82 69 F8 25 60 94 B9 D4 75 3A 87 CC E8 E6 01 D6 C9 A4 4D D5 1D 5C 75 A7 7C A9 84 C0 75",
"session id" : "79 91 31 B8 49 B9 86 47 72 F6 15 47 F2 2D FF 6E F5 20 96 22 84 58 C6 7C 0B A7 97 7B 13 8F CE 71",
"cipher suites" : "[TLS_AES_256_GCM_SHA384(0x1302), TLS_AES_128_GCM_SHA256(0x1301), TLS_CHACHA20_POLY1305_SHA256(0x1303), TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA9), TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA8), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_DHE_RSA_WITH_AES_256_GCM_SHA384(0x009F), TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCAA), TLS_DHE_DSS_WITH_AES_256_GCM_SHA384(0x00A3), TLS_DHE_RSA_WITH_AES_128_GCM_SHA256(0x009E), TLS_DHE_DSS_WITH_AES_128_GCM_SHA256(0x00A2), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_DHE_RSA_WITH_AES_256_CBC_SHA256(0x006B), TLS_DHE_DSS_WITH_AES_256_CBC_SHA256(0x006A), TLS_DHE_RSA_WITH_AES_128_CBC_SHA256(0x0067), TLS_DHE_DSS_WITH_AES_128_CBC_SHA256(0x0040), TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384(0xC02E), TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384(0xC032), TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256(0xC02D), TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256(0xC031), TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384(0xC026), TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384(0xC02A), TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256(0xC025), TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256(0xC029), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA(0xC00A), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA(0xC014), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA(0xC009), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA(0xC013), TLS_DHE_RSA_WITH_AES_256_CBC_SHA(0x0039), TLS_DHE_DSS_WITH_AES_256_CBC_SHA(0x0038), TLS_DHE_RSA_WITH_AES_128_CBC_SHA(0x0033), TLS_DHE_DSS_WITH_AES_128_CBC_SHA(0x0032), TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA(0xC005), TLS_ECDH_RSA_WITH_AES_256_CBC_SHA(0xC00F), TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA(0xC004), TLS_ECDH_RSA_WITH_AES_128_CBC_SHA(0xC00E), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_RSA_WITH_AES_128_GCM_SHA256(0x009C), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_RSA_WITH_AES_256_CBC_SHA(0x0035), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_EMPTY_RENEGOTIATION_INFO_SCSV(0x00FF)]",
"compression methods" : "00",
"extensions" : [
"status_request (5)": {
"certificate status type": ocsp
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
},
"supported_groups (10)": {
"versions": [x25519, secp256r1, secp384r1, secp521r1, x448, ffdhe2048, ffdhe3072, ffdhe4096, ffdhe6144, ffdhe8192]
},
"ec_point_formats (11)": {
"formats": [uncompressed]
},
"signature_algorithms (13)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, ed25519, ed448, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha224, rsa_sha224, dsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
},
"signature_algorithms_cert (50)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, ed25519, ed448, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha224, rsa_sha224, dsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
},
"status_request_v2 (17)": {
"cert status request": {
"certificate status type": ocsp_multi
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
}
},
"extended_master_secret (23)": {
<empty>
},
"session_ticket (35)": {
<empty>
},
"supported_versions (43)": {
"versions": [TLSv1.3, TLSv1.2]
},
"psk_key_exchange_modes (45)": {
"ke_modes": [psk_dhe_ke]
},
"key_share (51)": {
"client_shares": [
{
"named group": x25519
"key_exchange": {
0000: 83 7C C2 46 4E 67 C2 59 14 DD 91 F9 78 19 38 71 ...FNg.Y....x.8q
0010: C5 A1 83 2F 3F C9 06 EE 8D 05 E3 1F F9 FE BA 06 .../?...........
}
},
{
"named group": secp256r1
"key_exchange": {
0000: 04 20 65 93 7F DD D4 58 FE E2 3C 96 34 2B E1 FB . e....X..<.4+..
0010: 9F 15 0B 93 A4 42 24 F9 85 D2 76 1F 84 9E A2 AF .....B$...v.....
0020: 41 4B 4A BF 6F 2E F2 E8 2E 27 C8 10 7C FE 3C 46 AKJ.o....'....<F
0030: 94 37 2D EE 89 B9 61 80 5B 34 B5 A0 0E 48 3D 7E .7-...a.[4...H=.
0040: 98
}
},
]
}
]
}
)

-------------------



2. Server response
-------------------
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.908 PDT|ServerHello.java:888|Consuming ServerHello handshake message (
"ServerHello": {
"server version" : "TLSv1.2",
"random" : "61 37 6A EF 23 4A 78 B6 A9 F6 F6 07 18 5C 53 41 2C F2 18 D7 4B 88 30 C2 3B 8E 05 F0 CE FC DC B9",
"session id" : "E1 22 00 00 91 45 3F 0D 01 E9 D5 FE 3E 0D 5B 2A 06 AD 8D B0 BB 3F D9 9C 48 BD DF 68 63 39 B4 4A",
"cipher suite" : "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030)",
"compression methods" : "00",
"extensions" : [
"extended_master_secret (23)": {
<empty>
},
"renegotiation_info (65,281)": {
"renegotiated connection": [<no renegotiated connection>]
}
]
}
)
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.909 PDT|SSLExtensions.java:173|Ignore unavailable extension: supported_versions
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.909 PDT|ServerHello.java:984|Negotiated protocol version: TLSv1.2
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:192|Consumed extension: renegotiation_info
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:173|Ignore unavailable extension: server_name
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:173|Ignore unavailable extension: max_fragment_length
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:173|Ignore unavailable extension: status_request
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:173|Ignore unavailable extension: ec_point_formats
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:173|Ignore unavailable extension: status_request_v2
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:192|Consumed extension: extended_master_secret
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:173|Ignore unavailable extension: session_ticket
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:163|Ignore unsupported extension: supported_versions
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.910 PDT|SSLExtensions.java:163|Ignore unsupported extension: key_share
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:192|Consumed extension: renegotiation_info
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:163|Ignore unsupported extension: pre_shared_key
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: server_name
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: max_fragment_length
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: status_request
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: ec_point_formats
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: application_layer_protocol_negotiation
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: status_request_v2
javax.net.ssl|WARNING|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:215|Ignore impact of unsupported extension: extended_master_secret
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: session_ticket
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: supported_versions
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: key_share
javax.net.ssl|WARNING|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:215|Ignore impact of unsupported extension: renegotiation_info
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.911 PDT|SSLExtensions.java:207|Ignore unavailable extension: pre_shared_key
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.912 PDT|CertificateMessage.java:366|Consuming server Certificate handshake message (
"Certificates": [
"certificate" : {
"version" : "v3",
"serial number" : "29 59 EE EB AC A5 51 91 40 72 20 61 AB 77 86 2B",
"signature algorithm": "SHA256withRSA",
"issuer" : "CN=SSL_Self_Signed_Fallback",
"not before" : "2021-09-05 21:58:08.000 PDT",
"not after" : "2051-09-05 21:58:08.000 PDT",
"subject" : "CN=SSL_Self_Signed_Fallback",
"subject public key" : "RSA"}
]
)
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.915 PDT|ECDHServerKeyExchange.java:514|Consuming ECDH ServerKeyExchange handshake message (
"ECDH ServerKeyExchange": {
"parameters": {
"named group": "secp384r1"
"ecdh public": {
0000: 04 E3 B2 B7 6C C7 20 C3 40 1B 73 5B 75 71 9B 8E ....l. .@.s[uq..
0010: 8B A0 3B 1F 98 73 84 FF BE 03 00 9F 90 58 E8 A5 ..;..s.......X..
0020: D9 32 C8 78 BC 7D A8 98 74 6A F8 FC 80 3C 3B 16 .2.x....tj...<;.
0030: D3 9B 9E E3 A3 FD C8 34 37 F4 57 26 61 DB 12 E1 .......47.W&a...
0040: FF 1C 50 29 3D 48 03 FA F0 B8 B7 A5 02 BD 99 BD ..P)=H..........
0050: AF 19 EB 91 02 BF 8A BC 87 F6 17 9B 17 5D C2 DD .............]..
0060: 33 3
},
},
"digital signature": {
"signature algorithm": "rsa_pkcs1_sha256"
"signature": {
0000: 66 58 57 26 B0 3F 46 A6 1F F0 2C 50 DB 90 60 BF fXW&.?F...,P..`.
0010: A0 47 63 A1 00 A5 72 86 2D F7 0B ED ED 29 EE 5B .Gc...r.-....).[
0020: 36 86 A3 80 31 D3 72 D8 60 C0 DA 9C E1 7F 9D 25 6...1.r.`......%
0030: 90 A3 8F 19 DC A2 AD 59 2A 5B 3E 85 BD 1C 5F DE .......Y*[>..._.
0040: 1F 87 A7 FB 88 63 97 CE 70 60 E7 F2 5B E2 6F 69 .....c..p`..[.oi
0050: C5 2A 32 12 A2 09 8D 83 FC 0F B3 7A 90 36 B1 D4 .*2........z.6..
0060: 42 9D 60 B6 4F A9 5D BE 74 1A 8C 3D FC 9A A8 21 B.`.O.].t..=...!
0070: D9 30 1A 36 EB 68 36 35 06 B5 F9 29 F8 FB C3 8D .0.6.h65...)....
0080: E3 A4 B4 73 EA DF 57 23 49 4E D8 47 5B 05 E4 DE ...s..W#IN.G[...
0090: C4 40 93 42 E6 10 A3 49 E7 FB 8E 99 98 E5 8D 9E .@.B...I........
00A0: C0 96 94 F7 D9 1D 66 A3 6D AD BB 6A A6 0C 7B 20 ......f.m..j...
00B0: 84 4D 84 17 C6 5C BA AD 01 24 CE F0 5B 2D F7 76 .M...\...$..[-.v
00C0: F8 64 1F E8 CF 01 20 CB 60 E4 50 B5 A4 57 69 13 .d.... .`.P..Wi.
00D0: 41 43 F6 64 3B 4E 6A 1F DC 4A A5 A0 35 B9 9B 44 AC.d;Nj..J..5..D
00E0: 05 76 9A 52 D4 94 1A 53 8A 91 41 08 23 A2 61 05 .v.R...S..A.#.a.
00F0: 57 7B 11 D3 EA 1E 47 4A 04 E0 F7 F7 0D D0 C6 9C W.....GJ........
},
}
}
)
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.915 PDT|ServerHelloDone.java:151|Consuming ServerHelloDone handshake message (
<empty>
)
-------------------


3. Client next messages
-------------------
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.921 PDT|ECDHClientKeyExchange.java:400|Produced ECDHE ClientKeyExchange handshake message (
"ECDH ClientKeyExchange": {
"ecdh public": {
0000: 04 6A 64 AA 6F 96 E8 58 AC 09 A7 44 73 13 B2 23 .jd.o..X...Ds..#
0010: E5 EC C9 88 AE 6F 6C 02 6C F6 60 9B BE 80 2D D6 .....ol.l.`...-.
0020: 60 B3 B3 69 56 03 2A 5D 59 AA 44 41 4C 44 E0 DA `..iV.*]Y.DALD..
0030: 98 67 27 5B 6E FF ED 67 EB AB 72 21 8A A8 EE 97 .g'[n..g..r!....
0040: B3 33 95 6C 2B 37 D5 B4 50 97 C6 5F 0E B0 EF 69 .3.l+7..P.._...i
0050: 1E 2F 66 EA BF 4B A8 60 4B F7 DD 0F 73 28 E8 35 ./f..K.`K...s(.5
0060: 66 f
},
}
)
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.928 PDT|ChangeCipherSpec.java:115|Produced ChangeCipherSpec message
javax.net.ssl|DEBUG|6A|macbook2-thread-3|2021-09-07 06:36:47.928 PDT|Finished.java:398|Produced client Finished handshake message (
"Finished": {
"verify data": {
0000: 16 5C BF 89 5C 63 7E F0 5D 21 67 47
}'}
)


--------------------

4. At this point the client expects SQL Server response (perhaps final).
But the MS SQL Server instance goes silent - no traffic over teh socket, no TDS packets.
As a result the client never finishes the
SSL hanshake.

here is the log file with the SSL hadshake exchange - I can discuss and explain details, if needed