Infiniband under FreeBSD 11.0-CURRENT...

[Karl Pielorz]

--On 07 January 2015 13:45 +0000 Karl Pielorz <kpielorz_lst at tdx.co.uk> 
wrote:

> Ok, I'll set them to ib - and check the relevant modules are loaded,

Ok - sysctl'ing the ports to 'ib' - and opensm / ibstat now see them.

>> Also, please verify the relevant modules are loaded:
>> Kldstat -v | grep 'mlx4\|mlx4ib\|ibcore\|ipoib'

This gives:

    421 ipoib
    420 ibcore
    423 mlx4
    422 mlx4ib

Having configured the interfaces, and run the obligatory 'iperf' on them - 
the results aren't outstanding:

"
[  4] local 10.90.90.2 port 5001 connected with 10.90.90.1 port 64249
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-12.2 sec  1.12 MBytes   771 Kbits/sec

...

[  3] local 10.90.90.1 port 64249 connected with 10.90.90.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.7 sec  1.12 MBytes   879 Kbits/sec
"

Under 10.x I know it was better to disable connected mode - is there a way 
this can be done under 11.x - or should I see better performance than that 
using CM?

'ibstat' on the system returns:

"
CA 'mlx4_0'
        CA type: MT25418
        Number of ports: 2
        Firmware version: 2.6.0
        Hardware version: a0
        Node GUID: 0x001a4bffff0c328c
        System image GUID: 0x001a4bffff0c328f
        Port 1:
                State: Active
                Physical state: LinkUp
                Rate: 20
                Base lid: 1
                LMC: 0
                SM lid: 1
                Capability mask: 0x0251086a
                Port GUID: 0x001a4bffff0c328d
        Port 2:
                State: Initializing
                Physical state: LinkUp
                Rate: 20
                Base lid: 0
                LMC: 0
                SM lid: 0
                Capability mask: 0x02510868
                Port GUID: 0x001a4bffff0c328e
"

(Second port is 'Initialising' as I only have one instance of opensm 
running bound to the first port).

Thanks,

-Karl