[Bug 273051] ofed: ibv pingpongs not connecting through ipv4

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273051

            Bug ID: 273051
           Summary: ofed: ibv pingpongs not connecting through ipv4
           Product: Base System
           Version: CURRENT
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: rafal.michna@gmail.com

The fix for ibv pingpongs on ipv6 broke the functionality on ipv4.
https://reviews.freebsd.org/D37196

Easiest way to verify is to use an OS version which doesn't include the fix,
such as 13.1.
CURRENT, 13.2 and 12.4 show the same behavior as 13.1 after patching.

13.1 before applying the patch:

ipv4:

server # ibv_rc_pingpong --gid-idx=2 --ib-dev=irdma-ice0 --rx-depth=500
--iters=1 --size=1024
  local address:  LID 0x0001, QPN 0x000004, PSN 0x7a529b, GID 1234::89
  remote address: LID 0x0001, QPN 0x000004, PSN 0x130fe2, GID 1234::90
2048 bytes in 0.00 seconds = 5.83 Mbit/sec
1 iters in 0.00 seconds = 2811.00 usec/iter

client # ibv_rc_pingpong --gid-idx=2 --ib-dev=irdma-ice0 --rx-depth=500
--iters=1 --size=1024 100.0.0.89
  local address:  LID 0x0001, QPN 0x000004, PSN 0x130fe2, GID 1234::90
  remote address: LID 0x0001, QPN 0x000004, PSN 0x7a529b, GID 1234::89
2048 bytes in 0.00 seconds = 420.10 Mbit/sec
1 iters in 0.00 seconds = 39.00 usec/iter

ipv6:

server # ibv_rc_pingpong --gid-idx=2 --ib-dev=irdma-ice0 --rx-depth=500
--iters=1 --size=1024
  local address:  LID 0x0001, QPN 0x000005, PSN 0xaaf56a, GID 1234::89
^C

client # ibv_rc_pingpong --gid-idx=2 --ib-dev=irdma-ice0 --rx-depth=500
--iters=1 --size=1024 1234::89
  local address:  LID 0x0001, QPN 0x000005, PSN 0x29b905, GID 1234::90
Couldn't connect to 1234::89:18515

After applying the patch:

client # ibv_rc_pingpong --gid-idx=2 --ib-dev=irdma-ice0 --rx-depth=500
--iters=1 --size=1024 100.0.0.89
  local address:  LID 0x0001, QPN 0x000006, PSN 0x1607d8, GID 1234::90
Couldn't connect to 100.0.0.89:18515

client # ibv_rc_pingpong --gid-idx=2 --ib-dev=irdma-ice0 --rx-depth=500
--iters=1 --size=1024 1234::89
  local address:  LID 0x0001, QPN 0x000007, PSN 0x82bbce, GID 1234::90
  remote address: LID 0x0001, QPN 0x000006, PSN 0x680d13, GID 1234::89
2048 bytes in 0.00 seconds = 356.17 Mbit/sec
1 iters in 0.00 seconds = 46.00 usec/iter

Same behavior is present with mlx cards, so it's not a driver issue:

client # ibv_rc_pingpong --gid-idx=2 --ib-dev=mlx5_0 --rx-depth=500 --iters=1
--size=1024 200.0.0.89
  local address:  LID 0x0000, QPN 0x000193, PSN 0xe6daae, GID ::ffff:200.0.0.90
Couldn't connect to 200.0.0.89:18515

client # ibv_rc_pingpong --gid-idx=2 --ib-dev=mlx5_0 --rx-depth=500 --iters=1
--size=1024 200::89
  local address:  LID 0x0000, QPN 0x000194, PSN 0x047cf8, GID ::ffff:200.0.0.90
  remote address: LID 0x0000, QPN 0x000193, PSN 0x0054f6, GID ::ffff:200.0.0.89
2048 bytes in 0.00 seconds = 364.09 Mbit/sec
1 iters in 0.00 seconds = 45.00 usec/iter

Using different GIDs doesn't make a difference. On the working ip version, with
irdma any GID works and with mlx, only GID 2 does.

-- 
You are receiving this mail because:
You are the assignee for the bug.