Krping server port not getting released after completion of krping session

[Khoje, Anand]

Hi,

I am currently working on RoCE v1 support for Cavium BigBear adapters. Although, RoCE traffic seems to be working now, I have noticed that once we establish krping server at one end, using command :
echo "server,addr=192.168.11.11,port=9991,verbose" > /dev/krping

and run krping client from the peer system, using command :
echo "client,count=10,addr=192.168.11.11,port=9991,verbose" > /dev/krping

The session ends after sending 10 RDMA packets.
Here are the logs :
server ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqr
server ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrs
server ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrst
server ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrstu
server ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrstuv
server ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrstuvw
server ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrstuvwx
server ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrstuvwxy
server ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrstuvwxyz
server ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\134]^_`abcdefghijklmnopqrstuvwxyzA
DISCONNECT EVENT...
wait for RDMA_READ_ADV state 10
cq completion in ERROR state


But, if we attempt to re-establish the krping server in the same system with the same port number (i.e. 9991), we get an error saying :
rdma_bind_addr error -48 (address already in use?)

Meanwhile, I checked with Mellanox Connect X3 adapters, I see the same issue.
My question is, is this behavior normal? Or is there some issue with my setup? Is it due to some ibcore state machine being stuck in some state waiting for an event which has not been taken care of by the driver?

P.S:
Here is the output of netstat when a krping server is established  :
# netstat
Active Internet connections
Proto Recv-Q Send-Q      Local Address                   Foreign Address               (state)
tcp4       0            0             192.168.11.11.9991        *.*                                       CLOSED

This remains the same throughout, even after completion of krping session. I cannot see this information on the krping client system though.

Thanks,
Anand A. K.