Forum Discussion
RDMA (RoCE) Test Failed over different Subnet
Hello Everyone,
Let's say pSMBNIC1 and pSMBNIC2 are the names of the NICs to be used for RDMA on each node of a 3-node cluster. IP address assignments are as follows:
pSMBNIC1 = 192.168.207.31 (N1), 192.168.207.32 (N2), 192.168.207.33 (N3)
pSMBNIC2 = 192.168.206.51 (N1), 192.168.206.52 (N2), 192.168.206.53 (N3)
RDMA Test is Successful when I run for the following scenarios:
From pSMBNIC1 of N1 to pSMBNIC1 of N2 (192.168.207.32) and pSMBNIC1 of N3 (192.168.207.33)
From pSMBNIC1 of N2 to pSMBNIC1 of N1 (192.168.207.31) and pSMBNIC1 of N3 (192.168.207.33)
From pSMBNIC1 of N3 to pSMBNIC1 of N1 (192.168.207.31) and pSMBNIC1 of N2 (192.168.207.32)
From pSMBNIC2 of N1 to pSMBNIC2 of N2 (192.168.206.52) and pSMBNIC2 of N3 (192.168.206.53)
From pSMBNIC2 of N2 to pSMBNIC2 of N1 (192.168.206.51) and pSMBNIC2 of N3 (192.168.206.53)
From pSMBNIC2 of N3 to pSMBNIC2 of N1 (192.168.206.51) and pSMBNIC2 of N2 (192.168.206.52)
However, RDMA Test fails with "ERROR: RDMA traffic test FAILED: Please check ERROR: a) physical switch port configuration for Priority Flow Control. ERROR: b) job owner has write permission at 192.168.206.51 \C$" for the following scenarios:
From pSMBNIC1 of N1 to pSMBNIC2 of N2 (192.168.206.52) and pSMBNIC2 of N3 (192.168.206.53)
From pSMBNIC1 of N2 to pSMBNIC2 of N1 (192.168.206.51) and pSMBNIC2 of N3 (192.168.206.53)
From pSMBNIC1 of N3 to pSMBNIC2 of N1 (192.168.206.51) and pSMBNIC2 of N2 (192.168.206.52)
From pSMBNIC2 of N1 to pSMBNIC1 of N2 (192.168.207.32) and pSMBNIC1 of N3 (192.168.207.33)
From pSMBNIC2 of N2 to pSMBNIC1 of N1 (192.168.207.31) and pSMBNIC1 of N3 (192.168.207.33)
From pSMBNIC2 of N3 to pSMBNIC1 of N1 (192.168.207.31) and pSMBNIC1 of N2 (192.168.207.32)
This means that RDMA Tests are passing for the same subnets but failing when run across different subnets.
Is it normal? I have already enabled the PFC... But, even if the PFC is not enabled, then how tests are passing for the same subnet?
Please guide...
Thank you in anticipation.