SSL/TLS connection issue troubleshooting guide

[Shi_Ding]

Background:

Nowadays almost every service support connection over TLS to encrypt data in transit to protect data.

You may experience exceptions or errors when establishing TLS connections with Azure services. Exceptions are vary dramatically depending on the client and server types. A typical ones such as  "Could not create SSL/TLS secure channel." "SSL Handshake Failed", etc.

In this article we will discuss common causes of TLS related issue and troubleshooting steps.

How SSL/TLS connections are established:

Before we start, let us get to know how SSL/TLS connections are established. I know there are millions of articles out there explaining the same handshake process using different colors, styles and arrows, so here comes my version:

1. It is always client that starts a conversation. Client says "Hello, I would like to talk to you secretly by encrypting the messages. Here is my TLS version and a list CipherSuite I have on my hand. "
2. Server checks if itself supports same TLS version and go through server's own CipherSuite lists to see if there is any matching ones.
3. Server replies "Hello back, we can use the TLS version you sent and I find this CipherSuite from your list on my hand as well. Let's use this TLS version and CipherSuite. By the way here is my certificate (certificate chain) with my public key for you to check my identity."
4. Client review server's certificate, verify if the certificate is expired, if it is issued to the same server name client tried to access, if the certificate issuer is trustable, or if the certificate is ever revoked, etc. Once verification passed, client creates a random secret and encrypt with server's public key (derived from server certificate).
5. Client says "Alright let's use you picked cipher, here is secret key I encrypted with your public key. Let me know if you can understand."
6. The message sent from client can be only decrypted using server's private key which is known by server only and cannot be peeked by others. So yes server is able to decrypt the secret key.
7. Server replies "Let's encrypt using our own secret key and let's get our secret conversation start now!" (Normally server and client generate another key so called Master Key using the secrete key and agreed algorithm. Both server and client use the Master key for following message encryption and decryption. This step may be vary for different ciphers )
8. *#FSNV%^&BSJ}D#@#(#*;]#  (Client and server starts their own secret conversation encrypted with their agreed secret key which I don't even understand…)

SSL/TLS connection real case example:

Below is a real example showing how it looks like in network packet.  

If you capture network packet using Wireshark, Netmon or tcpdump, you can open the file in Wireshark.

Below is an example:

You may filter for “TLS” or “Client Hello” to locate the first TLS packet. 

1. Client Hello

2. Server Hello 

As you can see all elements needed during TLS connection are available in the network packet.

If you capture network packet for a not working case, you can compare with the above working one and find in which step it fails.

However in some cases, capturing network packet is not the best option or not even an option due to security reasons, for example many Azure PaaS service, such as Storage, Serivce Bus, etc are hosting in a shared tenant and we cannot capture the packet on server end. In those scenarios there are also a few handy tools available for troubleshooting.

In next blogs, I will introduce more TLS troubleshooting methods, common causes and corresponding solutions. Stay tuned.