How to force IB cards to come up in IB mode?

[Meny Yossefi]

Hi Justin,



It seems as the pre-boot configuration didn't work.

loader.conf doesn't work on all sysctl params AFAIK.



Is it a pre-boot only issue?

What is the current sysctl output for those ports?

Have you tried switching to IB after system was loaded?



If I remember correctly, the driver was set for Eth configuration by default.

Of course, this is reversible..



Regards,
Meny Yossefi | SW Manager | FreeBSD Team
Mellanox Technologies Ltd
Work: +972-74-7129121, Cell: +972-52-8379557



________________________________________

From: owner-freebsd-infiniband at freebsd.orgOn<mailto:owner-freebsd-infiniband at freebsd.orgOn> Behalf OfJustin Clift

Sent: Saturday, April 16, 2016 11:15:11 AM (UTC) Monrovia, Reykjavik

To: freebsd-infiniband at freebsd.org<mailto:freebsd-infiniband at freebsd.org>

Subject: How to force IB cards to come up in IB mode?



Hi all,



I'm trying to get a new FreeBSD 10.3-STABLE install to work well with my MHGH28-XTC (dual port ConnectX DDR) cards.



The problem is they're coming up in eth mode, not ib mode, and I can't seem to get that fixed. :(



>From boot:



  # ifconfig

  mlxen0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500

          options=6d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>

          ether 00:02:c9:00:ca:cd

          nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

          media: Ethernet autoselect

          status: no carrier

  mlxen1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500

          options=6d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>

          ether 00:02:c9:00:ca:ce

          nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

          media: Ethernet autoselect

          status: no carrier



My loader.conf file seems like it should be correct:



  # cat /boot/loader.conf

  sys.device.mlx4_core0.mlx4_port1="ib"

  sys.device.mlx4_core0.mlx4_port2="ib"



As background, this is a new install (coming from CentOS 6.x, where it all works well), so I know there's no problem with the cards.  They're plugged into a 24 port DDR switch (Voltaire ISR 9024D-M) using active optical cables.



Any ideas?



Regards and best wishes,



Justin Clift



--

"My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there."

- Indira Gandhi



_______________________________________________

freebsd-infiniband at freebsd.org<mailto:freebsd-infiniband at freebsd.org> mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband

To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe at freebsd.org<mailto:freebsd-infiniband-unsubscribe at freebsd.org>"