Forum Discussion

HasanHasib's avatar
HasanHasib
Copper Contributor
Sep 30, 2023

RDMA (RoCE) Test Failed over different Subnet

Hello Everyone,   Let's say pSMBNIC1 and pSMBNIC2 are the names of the NICs to be used for RDMA on each node of a 3-node cluster. IP address assignments are as follows: pSMBNIC1 = 192.168.207.31 (...
clustering
Hyper-V
storage
windows server
LeonPavesic's avatar
LeonPavesic
Silver Contributor
Oct 04, 2023

Hi HasanHasib,

It is not normal for RDMA tests to pass for the same subnets but fail when run across different subnets. There are a few possible reasons for this, including incorrect switch configuration, firewall blocking RDMA traffic, and incorrect RDMA settings on the nodes.

To troubleshoot the issue, you can try the following:

  1. Check the switch configuration to make sure that Priority Flow Control (PFC) is enabled on all ports connecting the nodes.
  2. Check the firewall configuration on all nodes to make sure that RDMA ports are open.
  3. Check the RDMA settings on all nodes to make sure that RDMA is enabled and that the same RDMA technology (iWARP or RoCE) is configured on all nodes.
  4. Try running the RDMA test again after making the necessary changes.


Please click Mark as Best Response & Like if my post helped you to solve your issue.
This will help others to find the correct solution easily. It also closes the item.


If the post was useful in other ways, please consider giving it Like.


Kindest regards,


Leon Pavesic
(LinkedIn)

Resources