API Management - Networking FAQs (Demystifying Series I)
Published Jul 01 2020 10:11 AM 20.8K Views
Microsoft

API Management – Networking FAQs

(Demystifying series I)

 

This article includes frequently asked questions about API Management while configuring it with Azure networking components such as VNET, NSG and route tables.

If you feel there are more topics to be covered under the demystifying series, let us know in the comments section.

 

FAQ Index:

Network Security Group:

User Defined Route:

VNET Peering:

 

Network Security Group (NSG):

 

1.    What is NSG? Purpose of NSG?

Azure network security group is used to filter network traffic to and from Azure resources in an Azure virtual network. A network security group contains security rules that allow or deny inbound network traffic to, or outbound network traffic from, several types of Azure resources. For each rule, you can specify source and destination, port, and protocol.

 

2.    Scenarios for using NSG with API Management

When an API Management is configured with a VNET, Network Security Group plays a vital role in allowing inbound and outbound traffic into the subnet in which API Management is deployed. You would not want all the resources within or outside the VNET to access the API Management and in that case you can configure a NSG at the API Management subnet level and allow/deny the desired ports/IPs.

 

3.    Scenario: The APIs in API Management shows loading, and the APIs never show up

Configuration:

In internal mode, an API Management service is hosted in a dedicated subnet of a VNET. This subnet must have a Network Security Group associated with it. Network Security Groups have default inbound and outbound security rules.

 
Symptoms:

Navigate to the APIs section under API Management and open browser developer tools. In the console section, you will see a similar message based on the screenshot below.

 

sathishkumartlb_2-1593602615579.png
Error message:

{"code":"ManagementApiRequestFailed","message":"Failed to connect to Management endpoint <your_apim_instance_name>.azure-api.net:3443 for a service deployed in a Virtual Network. Make sure to follow guidance at https://aka.ms/apim-vnet-common-issues.","details":null,"innerError":null}

 

Troubleshooting:

The error message clearly states that the API Management endpoint is not reachable, hence it is not able to fetch the resources of the APIs. Allowing Management endpoint traffic on port 3443 in inbound security rule of NSG should resolve the issue

 

sathishkumartlb_3-1593602684081.png
Additional Information:

 

4.    What are the mandatory inbound and outbound rules in NSG for API Management?

There are few mandatory ports which needs to be configured under the inbound and the outbound rules of the NSG for the proper functioning of API Management.

 

Source port

Destination port

Direction

Source Service tag

Destination service tag

*

3443

Inbound

API Management

Virtual Network

*

80, 443

Inbound

Internet

Virtual Network

*

443

Outbound

Virtual Network

Storage

*

1443

Outbound

Virtual Network

SQL

 

NSG Inbound rules: (default) 

sathishkumartlb_12-1594384906481.png

[The default NSG inbound rules]

 

NSG Inbound rules: (creation)

sathishkumartlb_13-1594384906497.png

[While creating an NSG inbound rule, make sure to select the service tag option under source to get the following parameters]

 

NSG Inbound rules: (custom)

sathishkumartlb_14-1594384906514.png

[The highlighted rules are the custom NSG inbound rules added. Note that the priority of these rules must be higher than that of the default rules]

 

 

NSG Outbound rules: (default)

sathishkumartlb_15-1594384906531.png

[The default NSG outbound rules]

 

NSG Outbound rules: (creation)

sathishkumartlb_16-1594384906545.png

[While creating an NSG outbound rule, make sure to select the service tag option under destination to get the following parameters]

 

NSG Outbound rules: (custom)

sathishkumartlb_17-1594384906565.png

 [The highlighted rules are the custom NSG outbound rules added. Note that the priority of these rules must be higher than that of the default rules]

 

The port 3443 is associated with the Management endpoint. A load balanced public IP address (VIP) will also be reserved to provide access to the Management service endpoint over port 3443. The public IP address is used only for control plane traffic to the Management endpoint over port 3443 and can be locked down to the API Management service tag.

The Azure Storage account connectivity is required in the outbound rules to store the API Management operation and data. This is the point where it is cached and retried for API Management.

 

Note: We don't need to explicitly allow outbound connection in NSG. 
It is allowed by default.
In case of any Deny rule in the outbound section, then the above mentioned two outbound ports must be allowed explicitly with a higher priority.

 

5.    Recommendation for NSG at the API Management level

All Incoming/outgoing traffic for the service goes through Azure Load balancer. So, at the minimum, it is required to allow incoming port 80, 443, 3443 and any outgoing ports that are required by their (APIM) backend. We recommend all ports as if you created a new API that connects to a backend over a non-standard port, that would fail unless the port is opened up for Azure Load Balancer. 

The security rule for Azure Load balancer must be configured as shown in the below screenshot.

 

sathishkumartlb_4-1593603477448.png

 

6.    What if I still face the management endpoint issue?

In case if you are still facing an issue though the NSG rules are configured as per the documentation, then the best practice at this point is to remove the NSG associated with the subnet under which the APIM is present. Save the changes at the VNET and then go to the respective APIM instance ->Virtual Network -> Click on ‘Apply Network Configuration’, once it is completed the network changes must be effective at the APIM level. 

If the issue is still occurring, then the cause is not the NSG, if you do not see the management endpoint issue once the NSG is disassociated, then it was the NSG configuration which was causing the issue and it is recommended to consult with your network team to understand what might be causing the issue. Kindly raise a support ticket if the issue persists. 

 

 

User Defined Route (UDR):

 

1.    What is UDR? Purpose of UDR?

Azure routes outbound traffic from a subnet based on the routes in a subnet's route table. Azure automatically creates system routes and assigns the routes to each subnet in a virtual network. You can't create system routes, nor can you remove system routes. There are scenarios where the system routes will not be sufficient to fulfill the requirement and in order to establish a connection/request-flow, we will need to over-ride the default routes using custom routes. You can achieve this using User Defined Routes concept.

 

2.    How can UDR be configured with API Management?

UDR can control only the outbound traffic from the subnet. When the API Management is configured in a subnet, route table can be associated with that subnet. The route table will have user-defined routes (UDR) to determine the next hop type for a particular address prefix.

 

sathishkumartlb_5-1593603523678.png

[The above screenshot shows the subnet in which the API Management is configured. The subnet contains the address range, NSG and a Route Table (UDR)].

 

3.    Why my APIs are not loading when the API Management is configured in a VNET?

Scenario:

The API management service is hosted in an internal mode and the APIs under the API Management are not showing up and keep loading forever. The below screenshot shows a sample of the issue scenario and what the console section of the browser developer tools read. [Refer to the screenshot from 3rd point under NSG]

 
Error message:

{"code":"ManagementApiRequestFailed","message":"Failed to connect to Management endpoint <your_apim_instance_name>.azure-api.net:3443 for a service deployed in a Virtual Network. Make sure to follow guidance at https://aka.ms/apim-vnet-common-issues.","details":null,"innerError":null}

 

Troubleshooting:

The first pointer to check if the subnet in which the API Management is hosted has NSG associated with it and inbound access for management endpoint (3443) is allowed.

If the issue is still occurring, then study the architecture and determine if the API management VNET is learning default route(0.0.0.0/0) from on-premise over ExpressRoute or BGP over IPsec or if forced tunneling is configured on the API management subnet. Creating a user-defined route for default (0.0.0.0/0) traffic in route table can force all traffic from API Management delegated subnet to flow through a Network Virtual Appliance or Virtual Network Gateway. To overcome the limitation, we need to add precise user-defined routes(UDRs) to steer management traffic back to Azure by setting the next hop type of these routes to "Internet".

Associate the route table to the subnet of the API Management and you can find the steps for the same here.

 
Additional Information:

Creating UDR in a Route Table

 

sathishkumartlb_6-1593603571088.png

 

1.     Search for ‘Route Table’ in Azure search bar.

2.     Click on Add and create a Route table.

 

sathishkumartlb_7-1593603605277.png

 

1.     Open the created route table, select Routes.

2.     Click on Add

3.     Enter Route name (any), address prefix, next hop type and its next hop address (if required).

 

4.    What happens when I have some user-defined routes configured in my route table and there is another user-defined route for 0.0.0.0/0 address prefix configured to the internet, which route will take precedence? 

There can be multiple user defined routes in a route table. Azure selects a route based on the destination IP address, using the longest prefix match algorithm. If the destination IP address is a part of any of the precise user-defined routes, Azure will use that user-defined route. For destination IP addresses not part of precise user-defined routes or system routes, Azure will use user-defined route for 0.0.0.0/0 address prefix to route traffic.

 

sathishkumartlb_8-1593603642160.png

[The above picture illustrates three routes, which contains two custom user defined routes (NextHop and Custom-VNET) and one default user defined route (the ‘default’ route is one of the system route, but since it is defined explicitly by user, it becomes a default user defined route). For the custom user defined address prefix, the next hop is set according to the route while all the requests from other address prefix are handled by the default user defined route.]

 

VNET Peering:

 

1.    What is VNET Peering? Purpose of VNET Peering?

Virtual network peering enables you to seamlessly connect Virtual Networks in Azure. The virtual networks appear as one for connectivity purposes. The traffic between virtual machines uses the Microsoft backbone infrastructure. Like traffic between virtual machines in the same network, traffic is routed through Microsoft's private network only.

Azure supports the following types of peering:

  • Virtual network peering: Connect virtual networks within the same Azure region.
  • Global virtual network peering: Connecting virtual networks across Azure regions.

The benefits of using virtual network peering, whether local or global, include:

  • A low-latency, high-bandwidth connection between resources in different virtual networks.
  • The ability for resources in one virtual network to communicate with resources in a different virtual network.
  • The ability to transfer data between virtual networks across Azure subscriptions, Azure Active Directory tenants, deployment models, and Azure regions.
  • The ability to peer virtual networks created through the Azure Resource Manager.
  • The ability to peer a virtual network created through Resource Manager to one created through the classic deployment model. To learn more about Azure deployment models, see Understand Azure deployment models.
  • No downtime to resources in either virtual network when creating the peering, or after the peering is created.

Network traffic between peered virtual networks is private. Traffic between the virtual networks is kept on the Microsoft backbone network. No public Internet, gateways, or encryption is required in the communication between the virtual networks (exception based on resources used/configured in the VNET).

 

2.    How different is VNET Peering for API Management than connecting from the same VNET?

Consider a scenario where the API Management is configured in the internal mode of a VNET. The API Management consumes an application in the same VNET as its backend for one of the API hosted in it. Now, you have an Application which is configured in a different VNET, but you need to configure the same Application as the backend for another API hosted in the same API Management. It is not an ideal method to deploy another Premium SKU instance of API Management in the Backend Application VNET which is present in the same subscription and in the same region (only different VNETs).

 

Peering of VNETs will prove to be handy in this scenario. You can peer both the VNETs using VNET peering, now both the VNETs tend to act as a one whole VNET. So, it is possible to access the resources present in one VNET by the resources from the other via a private connection (private IP).Hence API Management will be able to import an Application from the other VNET as an API and work as if it was from the same VNET, thanks to VNET peering. Below provided is a pictorial representation of how VNET peering works and encapsulates resources from both VNETs to act as if a single VNET.

 

sathishkumartlb_9-1593603684136.png

[The NSG and route table associated with subnets of VNET 1 will be effective only within VNET 1, even though if there is a VNET peering with VNET 2 (the same applies for VNET 2). The resources under both the VNETs tend to act as if it is present under a single VNET]

 

3.    Unable to load APIs in Azure portal for API Management as the NSGs and Routes are publicized in VNET Peering

Scenario:

In this scenario, the API in API Management is configured to a resource/application as a backend from a peered VNET (which is usually internal). Now, in the Azure portal under API Management the APIs are not loading (ie) it keeps shows the loading screen forever when clicked on APIs under API Management.

 
Error:

The browser developer tools console section reads something like, the Management endpoint is not reachable or not found.

 
Troubleshooting:

As a primary step, we navigate to the NSG associated with the API Management subnet and see that the port 3443 is allowed under the inbound rules and yet the issue is existing. Once the VNET peering is removed for the VNET which contains the API Management in a subnet, the issue will get resolved. But, at this point the backend of one of the API in API Management is present in the other VNET (peered one). Hence all the request to the API fails as it will not be able to reach the backend. Removing VNET peering cannot be considered as a solution at this point.

 
Option #1

Add a user-defined default route for 0.0.0.0/0 address prefix with next hop type as Internet

This will force all outbound traffic from the API Management subnet directly to the internet. Even if on-premises is advertising 0.0.0.0/0 address prefix to API Management VNET, the user-defined route (default and custom) will still have more precedence, and the Management calls will be allowed back out to the internet directly.  

 

sathishkumartlb_0-1594128106420.png

[The above screenshot shows the user defined default route added to the route table]

 
Option #2

When we connect VNETs using VNET Peering but the routes are publicized by the connected network and will affect the connecting network.

We may be able to correct this by associating a route table to the API Management subnet and add the required Control Plane IP addresses.

 

sathishkumartlb_1-1594128266295.png

[The APIM is in East US region and its respective control plane IP is '52.224.186.99' which has been configured as a user defined route in the route table and the next hop is set to internet]

 

4.    What is the limitation of VNET Peering with API Management?

Connectivity from a resource in a globally peered VNET in another region to API Management service in Internal mode will not work due to platform limitation. For more information, see Resources in one virtual network cannot communicate with Azure internal load balancer in peered virt...

 

Related Links:

Continue Reading Part II : APIM - Networking FAQs (Demystifying series II)
8 Comments
Microsoft
  • Perfect.. nicely written :)

Nice Resource - Good work

Microsoft

Good article. Liked it more as it was concise and straight to point. Thanks

Copper Contributor

Thanks for the article. It is nicely explained.

I am facing the same issue with APIM after adding UDR pointing to Azure pipeline, that API's do not load and receive an error  "Failed to connect to management endpoint servicename.management.azure-api.net:3443 for a service deployed in a Virtual Network. Make sure to follow guidance at https://aka.ms/apim-vnet-common-issues.management endpoint."

I have ensured that the NSG has inbound and outbound rules set as mentioned in this article and have added routes - global and region speicific ip address with destination as internet and 0.0.0.0 pointing to the Azure firewall in UDR (https://techcommunity.microsoft.com/t5/azure-paas-developer-blog/api-management-networking-faqs-demy...).

 

Please suggest.

@cchat Thanks for your comment. 

It seems required NSG rules and route table entries are in place but still you are getting 'Failed to connect to management endpoint...'.

The best way to isolate the issue is to remove the NSG and route table from the subnet.

Start adding the route table entries one by one and observe the behavior. 

Copper Contributor

Hello,

 

We have implemented APIM in our Azure VNET.   We have an API (very simple test API) hosted in our AKS instance also in the same Azure VNET.   We have been working with support with out any success.   We cannot get APIM to successfully talk with our test API.   Has anyone else had this problem?   Any suggestions on what we might try to determine the root cause.

 

Bryan

@BryanDGarcia 

Please follow the below action plan.

1. Review APIM subnet NSG and route table and make sure that you have allowed required ports - https://docs.microsoft.com/en-us/azure/api-management/virtual-network-reference?tabs=stv2#required-p...

2. Make sure that you can invoke the API and get expected response from a POD or any VM from another subnet(associate the same NSG and route table of APIM subnet).

2. If still problem persists, please create support ticket with APIM. We can help you validate and identify the issue. 

Copper Contributor

@karthikeyanRamasamy

We have validated the API is fully accessible from our VNet.   The traffic flow we are trying to implement is

 

Internet -> Firewall -> Web Application Firewall -> APIM -> API (hosted in Kubernetes)

 

We know everything works Internet all the way to the VNET where APIM is located as we do the same thing as above but with Ocelot (Open Source API gateway) in place of APIM.   This is currently in production and works well.   We are trying to move to APIM since it has more advanced capabilities.   

 

We have been actively engaging with support (Professional Direct), but to date we have not had any success.

 

Thank you

Bryan

Version history
Last update:
‎Sep 23 2020 12:55 AM
Updated by: