FreeBSD 11 Infiniband performance / usable?

[Hans Petter Selasky]

On 01/10/15 18:59, Karl Pielorz wrote:
>
> Hi,
>
> I've now got my MT25418's running 'back to back' on a couple of FreeBSD
> 11 boxes [amd64 Haswell based] - but the performance seems rather poor
> using them for IPoIB? (iperf sites around 2Mbyte/sec on defaults -
> rising to 98Mbyte/sec if I set an MTU of 4k) - default on the card was 64k.
>
> Disabling IBOIB_CM in the kernel doesn't seem to make any difference
> (under 10.0 this made a huge difference).
>
> Also - the two boxes have lost site of each other now [I had this
> problem previously with 10.0 - one of the reasons why I was looking at 11].
>
>
> What's the current state of IB support in FreeBSD 11? - I know a lot of
> work has been done on this - but is it usable yet?
>
> The two machines both have MT25418's fitted running 'back to back' with
> one CX4 cable between them, and opensm running on them (after running
> the kernel sysctl's to switch the ports from 'eth' mode to 'ib').
>
> Cheers,

Hi,

IPoIB needs to be optimized to shuffle more packets per request to and 
from the kernel basically. Currently that is not done. Are you planning 
to use IPoIB for real workloads?

--HPS