Ib0 not working unless i unload ipoib and load it again.
Mihai Vintila
unixro at gmail.com
Sun Jun 14 18:40:05 UTC 2015
Another strange issue. Is that i have:
/boot/loader.conf
mlx4_core="YES"
ibcore_load="YES"
mlxen_load="YES"
mlx4ib_load="YES"
ipoib_load="YES"
and in sysctl.conf
sys.device.mlx4_core0.mlx4_port1=ib
Only when it boots ib0 is not functional. From what i've noticed if
after mlx4_core0 is up i remove ipoib module and add it again it's working.
Bellow is a small output from dmesg in which i unload/load ipoib at boot
in rc.local and interface still doesn't work and after port is up i do
it manually and it starts working:
ib0: Attached to mlx4_0 port 1
ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2>
on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
ib0: link state changed to DOWN
ib0: Attached to mlx4_0 port 1
mlx4_core0: mlx4_ib: Port 1 logical link is up
ib0: link state changed to DOWN
ib0: Attached to mlx4_0 port 1
ib0: link state changed to UP
in ifconfig only difference is a new lladdr for ib0.
Best regards,
Vintila Mihai Alexandru
On 6/13/2015 9:59 PM, Hans Petter Selasky wrote:
> On 06/13/15 20:43, Mihai Vintila wrote:
>> Hi,
>> I've posted in the stable mailing list but i guess this is where i
>> should ask.
>> Can you tell me what is the proper way to get subnet manager working on
>> FreeBSD 10.1 ?
>> I've used RELEASE branch to compile opensm and drivers from Mellanox but
>> i get:
>> ibportstate -G 0xe61d2dfffe0a2130 1 enable
>> ibpanic: [868] madrpc_init: can't open UMAD port ((null):0): (m)
>>
>> opensm
>> -------------------------------------------------
>> OpenSM 3.3.1
>> Reading Cached Option File: /etc/opensm/opensm.conf
>> Command Line Arguments:
>> Log File: /var/log/opensm.log
>> -------------------------------------------------
>> OpenSM 3.3.1
>>
>> Entering DISCOVERING state
>>
>> Using default GUID 0x0
>>
>> Error from osm_opensm_bind (0x2A)
>> Perhaps another instance of OpenSM is already running
>> Exiting SM
>>
>>
>>
>> kldstat
>> Id Refs Address Size Name
>> 1 27 0xffffffff80200000 1756638 kernel
>> 2 1 0xffffffff81957000 2681e8 zfs.ko
>> 3 2 0xffffffff81bc0000 6780 opensolaris.ko
>> 4 2 0xffffffff81bc7000 17d60 nvme.ko
>> 5 1 0xffffffff81bdf000 4210 nvd.ko
>> 6 1 0xffffffff81e11000 357f ums.ko
>> 7 3 0xffffffff81e15000 37656 mlx4.ko
>> 8 2 0xffffffff81e4d000 4e348 ibcore.ko
>> 9 1 0xffffffff81e9c000 2f72 toecore.ko
>> 10 1 0xffffffff81e9f000 238fa mlx4ib.ko
>> 11 1 0xffffffff81ec3000 12482 mlxen.ko
>>
>> Driver that i'm using:
>> mlx4_core0: <mlx4_core> mem 0xc7400000-0xc74fffff,0xc5000000-0xc57fffff
>> irq 32 at device 0.0 on pci2
>> mlx4_core: Initializing mlx4_core: Mellanox ConnectX VPI driver v2.1.6
>> (Jun 12 2015)
>>
>> mlx4_core0: mlx4_ib_add: allocated counter index 1 for port 1
>> mlx4_en mlx4_core0: Activating port:1
>> mlxen0: Ethernet address: e4:1d:2d:0a:21:30
>> mlx4_en: mlx4_core0: Port 1: Using 12 TX rings
>> mlxen0: link state changed to DOWN
>> mlx4_en: mlx4_core0: Port 1: Using 8 RX rings
>> mlx4_en: mlxen0: Using 12 TX rings
>> mlx4_en: mlxen0: Using 8 RX rings
>> mlx4_en: mlxen0: Initializing port
>>
>
> Hi,
>
> Did you check your ports are configured for "ib" and not "eth" ?
>
> For example:
> sysctl sys.device.mlx4_core0.mlx4_port1=ib
>
> --HPS
>
More information about the freebsd-infiniband
mailing list