Cloud Management Gateway - Inbound Rule for Port 8443

[NandanSheth]

Hello! My name is Nandan Sheth, and I am a part of Microsoft’s Customer Success Unit based out of Dublin, Ireland. I have been helping customers set up the Cloud Management Gateway for a few years now, but recently an organization with 40000+ users asked me a question that I haven’t given much thought to. When you set up the Cloud Management Gateway using a Virtual Machine Scale Set, one of the resources created in Azure is the Network Security Group. The Network Security Group has an inbound rule for port 8443.

Why is this rule created and what is it needed for?  

Figure 1: Network Security Group showing the inbound rule for port 8443.

After searching a bit, I realized that this information is not properly documented and unless you really start digging into all the configuration that is performed to create the Cloud Management Gateway, there’s no way to explain this rule. Once you have had a dig around though, it’s not so complicated after all. In this blog post, I want to try and explain the purpose of this inbound rule for port 8443. 

Let’s start with a review of the ports that are used to facilitate the Cloud Management Gateway connections.

The ports used for connections to the Cloud Management Gateway are documented on this link. Port 8443 is not documented at all. 

Client	Protocol	Port	Server	Description
Service connection point	HTTPS	443	Azure	CMG deployment
CMG connection point (virtual machine scale set)	HTTPS	443	CMG service	Protocol to build CMG channel to only one VM instance
CMG connection point (virtual machine scale set)	HTTPS	10124-10139	CMG service	Protocol to build CMG channel to two or more VM instances
CMG connection point (classic cloud service)	TCP-TLS	10140-10155	CMG service	Preferred protocol to build CMG channel
CMG connection point (classic cloud service)	HTTPS	443	CMG service	Fall back protocol to build CMG channel to only one VM instance
CMG connection point (classic cloud service)	HTTPS	10124-10139	CMG service	Fall back protocol to build CMG channel to two or more VM instances
Client	HTTPS	443	CMG	General client communication
Client	HTTPS	443	Blob storage	Download cloud-based content
CMG connection point	HTTPS or HTTP	443 or 80	Management point	On-premises traffic, port depends upon management point configuration
CMG connection point	HTTPS or HTTP	443 or 80 / 8530 or 8531	Software update point	On-premises traffic, port depends upon software update point configuration

With a Virtual Machine Scale Set, the Cloud Management Gateway service can be scaled up or down as needed. When the Cloud Management Gateway service has a single instance, all connections from the Cloud Management Gateway Connection Point to the Cloud Management Gateway Service are set up using port 443. We can validate this by running netstat -aon on the server hosting the Cloud Management Gateway Connection Point.

You can identify the public IP address by clicking into the Virtual Machine Scale Set. It is displayed in the Overview section. I have the public IP address hidden for security measures. 

Figure 2: Overview of the Virtual Machine Scale Set to identify the public IP Address.

netstat -aon shows connections to this IP address.

Figure 3: Output of netstat -aon with a single VM instance.

If I increase the number of instances in my Cloud Management Gateway service, and run netstat -aon again, I see the following details being returned:

Figure 4: Figure 3: Output of netstat -aon with multiple VM instances.

From the Cloud Management Gateway Connection Point, the connections are now being set up to ports 10124 and 10125. What happens to these connections on the Azure side? How is the traffic processed from these public ports?

The inbound rule for port 8443 starts coming into play when we increase the number of instances in the Cloud Management Gateway service. Because we have multiple virtual machines in Azure now, traffic goes through the Network Load Balancer. This resource is also created during the Cloud Management Service setup. Let’s take a look at the Network Load Balancer in Azure.

• From the Azure portal, click into the resource group that hosts the Cloud Management Gateway service.
• Then, click into the Load Balancer resource.
• Once you are on the Load Balancer blade, click on Frontend IP Configuration.

Figure 5: Options under Network Load Balancer.

• In the details for the Frontend IP Configuration, there should be a single configuration – loadBalancerFrontEndWeb. Click into this configuration.
• You should now see all the load balancing rules, Network Address Translation pools and inbound Network Address Translation rules.

Figure 6: Network Load Balancer Frontend Configuration showing Load Balancing configuration.

• As you increase the number of instances in your Cloud Management Gateway service, you will see that the number of inbound Network Address Translation rules increases. The screenshot above was taken for a Cloud Management Gateway service with two instances.
• Natpoolfe2.0 to 2.x show that traffic to the public IP address on ports 10124 to 10139 is being translated to port 8443. The screenshot below shows details for natpoolfe2.0 which translates 10124 to 8443. Similarly, natpoolfe2.1 will translate 10125 to 8443 and so on.

Figure 7: Network Mapping between port 10124 and 8443.

• Natpoolfe.0 and natpoolfe.1 are unrelated and are used to translate Remote Desktop Protocol traffic to port 3389 and are not relevant here.
• We can look at the overall topology for the Cloud Management Gateway service and this is how we can explain the Network Security Group inbound rule for port 8443.

Figure 8: Network Topology of Cloud Management Gateway with description of all the components.

It is important to note that the inbound rule on Network Security Group is required to ensure that this translated traffic is allowed on to the virtual network that the Cloud Management Gateway service is attached to. If this rule is deleted, connections to the virtual machines in the Virtual Machine Scale Set may fail. On the public IP address, there is no listener for port 8443. This can be further validated using a simple PowerShell command as shown below. Test-NetConnection on port 433, 10124, 10125… completes successfully. However, Test-NetConnection on port 8443 fails.

For more details on Network Security Groups, refer to these links: 

Azure network security groups overview | Microsoft Learn 

Network security group - how it works | Microsoft Learn 

Hopefully, this clarifies the use of the Network Security Group inbound rule for port 8443 in case it is something you are asked to investigate.