VMs on the same VNET can not talk to each other

[Gregory Suvalian]

Not sure what is exactly happening since it was always working before but VMs on different subnets within the same virtual network with no NSGs or firewalls between them can not talk to each other. Ping is failing as well as any other sort of communication. Firewalls are disabled on both sides. All machines have access to internet. Communication was tried using IP addresses and not names. Both ping as well as TCP based tests were used.

[#2]

There are couple of things that could be at work here.

When you say 'talk to each other', what do you mean?

You mentioned the VMs have internet connection so i am assuming you are able to access them via RDP. Can you Port Query the RDP port from one VM to another? if so whats the result?

[#3]

Mean I can not ping between them or can not open TCP connection between them. Yes internet and routing to proper Internet works fine, just communication between them is not. I'm using IP addresses for communication, firewalls on hosts are disabled.

PS C:\Windows\system32> Test-NetConnection 10.0.0.4 -port 3389
WARNING: Ping to 10.0.0.4 failed -- Status: TimedOut
WARNING: TCP connect to 10.0.0.4:3389 failed


ComputerName           : 10.0.0.4
RemoteAddress          : 10.0.0.4
RemotePort             : 3389
InterfaceAlias         : Ethernet 2
SourceAddress          : 10.0.1.5
PingSucceeded          : False
PingReplyDetails (RTT) : 0 ms
TcpTestSucceeded       : False

 The same test running against VM on local subnet works fine as expected

PS C:\Users\cloudadmin> test-netconnection 10.0.1.5 -port 3389


ComputerName           : 10.0.1.5
RemoteAddress          : 10.0.1.5
RemotePort             : 3389
InterfaceAlias         : Ethernet 7
SourceAddress          : 10.0.1.6
PingSucceeded          : True
PingReplyDetails (RTT) : 3 ms
TcpTestSucceeded       : True

[#4]

OK.

Let do a 'Route Print' and 'ipconfig /all' on 10.0.0.4 to confirm there is no custom route diverting traffic on the OS level.

[#5]

===========================================================================
Interface List
 14...00 0d 3a 02 8d 68 ......Microsoft Hyper-V Network Adapter #2
  1...........................Software Loopback Interface 1
 15...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0         10.0.0.1         10.0.0.4    261
         10.0.0.0        255.0.0.0         On-link          10.0.0.4    261
         10.0.0.4  255.255.255.255         On-link          10.0.0.4    261
   10.255.255.255  255.255.255.255         On-link          10.0.0.4    261
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link          10.0.0.4    261
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link          10.0.0.4    261
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0         10.0.0.1  Default 
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
  1    306 ::1/128                  On-link
 14    261 fe80::/64                On-link
 14    261 fe80::8d2a:8b0e:c4ad:807b/128
                                    On-link
  1    306 ff00::/8                 On-link
 14    261 ff00::/8                 On-link
===========================================================================
Persistent Routes:
  None

Windows IP Configuration

   Host Name . . . . . . . . . . . . : core-dc01
   Primary Dns Suffix  . . . . . . . : ecomply.local
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : ecomply.local

Ethernet adapter Ethernet 2:

   Connection-specific DNS Suffix  . : 
   Description . . . . . . . . . . . : Microsoft Hyper-V Network Adapter #2
   Physical Address. . . . . . . . . : 00-0D-3A-02-8D-68
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::8d2a:8b0e:c4ad:807b%14(Preferred) 
   IPv4 Address. . . . . . . . . . . : 10.0.0.4(Preferred) 
   Subnet Mask . . . . . . . . . . . : 255.0.0.0
   Default Gateway . . . . . . . . . : 10.0.0.1
   DHCPv6 IAID . . . . . . . . . . . : 335547706
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-21-F4-ED-77-00-15-5D-0A-1D-DE
   DNS Servers . . . . . . . . . . . : ::1
                                       10.0.0.4
                                       10.0.0.5
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter isatap.{4EA8D25B-523C-4E10-B0A9-3A7FE5508DC1}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : 
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

[#6]

2 things that stands out in OS configuration.

1. The persistent route should be needed

2. Static IP for Azure VM should be setup from the portal and not in the guest OS

Also did the issue start after a change to the environment? (joined domain, etc...)

Since the issue is at the subnet level confirm no subnet NSG or User Define routes on subnet.

[#7]

Neither persistent route no static IP is required to route between subnets. They shall work per documentation out of the box with no additional changes.
There is no NSGs setup on any level, there is no firewalls on either side.
I'm pretty sure something is broken in Azure specifically for my VNET setup and not something specific to VMs which I put in.

[#8 · Accepted solution]

In that case i suggest you file a ticket so the network can be checked in the back end.

[#9]

The answer in this area is quet easy. Since the last Azure Network changes and the introduction of "Vnet Peering" each subnet og a VNET has his own NSG and route table assignement. so in this case when you work only in one VNET you Have to create a custom route table an have to set the "virtual Network Gateway route Propagation" to "enabled". So all subnets have a container to publish there gateway and next-Hop configuration. configure each subnet with the same route table and Problem is solved