Azure Firewall in force tunnel configuration dropping port 53 traffic?

[John Wildes]

Hello All

I have a hub and spoke architecture where an azure firewall is in the hub and is our chosen NVA for traffic management.  With appropriate routing tables in place all traffic passes through the Azure firewall to our on prem datacenter and back.  

However, at the moment we are using on prem datacenter DNS servers for name resolution.  When these requests go through the firewall I see ALLOW statements on the firewall logs, but the traffic gets dropped.  I cannot do name resolution using an on prem DNS server.  

I can't even psping a DNS server (eg psping -t 10.x.x.x:53 the packets say ALLOW from the VM doing the PSPING to the address on prem through the firewall logs, but I get NO response.  I can psping the same DNS server on another port (3389) for example, and see ALLOW statements in the Azure firewall.

With no route tables in place, just the VPN connection on prem, I am able to PSPING the DNS servers on port 53 and able to get name resolution working.  It's only when I have the Azure firewall as the NEXT hop for any traffic that the port 53 traffic gets dropped.

if I configure the Azure firewall to be DNS proxy, which is in PREVIEW, and use only the Azure DNS resolver (not my on prem DNS) I'm able to resolve internet addresses, but not on prem DNS names. 

The only rule I have in place on the Azure firewall is a network rule, that allows sources to all destinations on all ports.  I have also configured an all sources to all destinations rule on port 53 traffic, and still get no PSPING or NSLOOKUP response from clients going through the firewall.

We've destroyed and recreated this firewall twice now, same results.

Please tell me I'm missing something simple, and the problem is with my configuration and NOT azure firewall.  I can find NO documentation that the azure firewall will drop traffic on port 53 if it sees it.

[#2]

Have you found any solutions to this? I am running into the exact same issue.

Thanks

[#3]

@xmnj40 

So what ended up being the issue in my environment is that the setting for "Propagate Gateway Routes was set to NO for the UDR on my GatewaySubnet and on my AzureFirewallSubnet.  My AzureFirewallSubnet (the private side) default route was 0.0.0.0/0 to Virtual Network Gateway.  The UDR used on the subnets on my peered VNETs would route all traffic 0.0.0.0/0 to AzureFirewall-privateIP. 

We use one primary VNET as the hub or transit vnet.  Everything was peered to this.  In short ensure that there is a UDR on the gateway subnet, and that the route propagation is turned on that UDR, and the route propagation is turned on the AzureFirewallSubnet.   It took Microsoft 4 months to close this ticket on my azure subscription with essentially "unsupported configuration" after reviewing with the product group at least 4 times.  Their only statement of "supported configuration" is found here.

https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-udr-overview#border-gateway-protocol 

"Connectivity with VPN connections is achieved using custom routes  with a next hop type of Virtual network gateway. Route propagation should not be disabled on the GatewaySubnet. The gateway will not function with this setting disabled."

Routes tell interfaces where to send packets.  Packets need an IP address, route tables deal with IP addresses.  When I attempt to reach 10.1.1.1 with a browser, it sends that traffic on port 80 to 10.1.1.1 when I use MSTSC it sends that traffic on 3389.  It doesn't change the route based on the port??? AND this configuration works when there's no Azure Firewall between the source and the destination.  Only when there is an Azure Firewall between the source and destination will the traffic disappear (and report in the logs that it has been accepted and forwarded), if that traffic is travelling over port 53.

[#4]

@John Wildes 

Thank you very much for this!

Been banging my head against the wall with this, and support has not been helpful...

[#5]

When enabling route propagation on UDR (GatewaySubnet), would you know if Microsoft changes any parameters related to MTU (Path MTU Discovery)?

I understand that this option should be enabled for use with BGP.