Does mlxen work on FreeBSD 10.1 with ConnectX-3

[Lewis, Fred]

Hi FreeBSD IB folks,

I'm trying to bring up a Mellanox Ethernert Interface and it is showing as "no carrier."
The cables were swapped out with a known good one and it didn't seem to make any difference.
The remote peer port is admin UP.

I've built the KLDs using the latest FreeBSD OFED stuff (v2.1.5).
The build machine is different from the machine that I'm loading the KLDs on.
The build machine does not have any OFED options or devices installed.

[mlxen]$ sysctl kern.conftxt | grep  -i ofed
[mlxen]$

I built the KLDs as follows:
cd mlx4 ; make clean cleandepend; make
cd mlxen; make clean cleandepend; make

I then copied the KLDs (mlx4.ko and mlxen.ko) to /boot/kernel on the target machine
and did a kldload. The modules loaded fine as far as I can tell and mlxen0 shows up in
the interface list.

mlx4_core0 at pci0:5:0:0:  class=0x028000 card=0x100315b3 chip=0x100315b3 rev=0x00 hdr=0x00
    vendor     = 'Mellanox Technologies'
    device     = 'MT27500 Family [ConnectX-3]'
    class      = network

root at mau-da-27-4-3:/boot/kernel # kldload mlx4.ko; kldload mlxen.ko
mlx4_core0: <mlx4_core> mem 0x93a00000-0x93afffff,0x383fff000000-0x383fff7fffff irq 42 at device 0.0 on pci5
<6>mlx4_core: Mellanox ConnectX core driver v2.1 (Feb  4 2015)
<6>mlx4_en: Mellanox ConnectX HCA Ethernet driver v2.1 (Feb  4 2015)
mlxen0: Ethernet address: 00:1e:67:81:40:61
<Feb  4 14:23:18 4>mau-da-27-4-3 kemlrnel: mlx4_en: mx4lx4_core0: Port _e1: Using 24 TX rn:ings
 mlx4_core0: Port 1: Using 24 TX rings
<4>mlx4_en: mlx4_core0: Port 1: Using 16 RX rings
<Feb  4 14:23:18 4>mau-da-27-4-3 kemlrnel: mlx4_en: mx4lx4_core0: Port _e1: Using 16 RX rn:ings
 Ub  4 14 m:23:18 mau-da-27lx-4-3 kernel: mlxen4_en: mlxen0: Us0:ing 24 TX rings
sing 24 TX rings
<4>mlx4_en: mlxen0: Using 16 RX rings
Feb  4 14:23:18 <4mau-da-27-4-3 ke>mrnel: mlx4_en: mlxlxen0: Using 16 4_RX rings
en: mlxen0: Initializing port
Feb  4 14:23:19 mau-da-27-4-3 kernel: mlx4_en: mlxen0: Initializing port

root at mau-da-27-4-3:/boot/kernel # kldstat -v | grep 'mlx4\|mlxen'
 9    2 0xffffffff81c6c000 37816    mlx4.ko (/boot/kernel/mlx4.ko)
                494 mlx4
10    1 0xffffffff81ca4000 125e2    mlxen.ko (/boot/kernel/mlxen.ko)
                495 mlxen

ifconfig -a
<snip>
mlxen0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE>
        ether 00:1e:67:81:40:61
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier

root at mau-da-27-4-3:/boot/kernel # sysctl -a | grep mlx4
hw.mlxen0.conf.device_name: mlx4_core0
dev.mlx4_core.%parent:
dev.mlx4_core.0.%desc: mlx4_core
dev.mlx4_core.0.%driver: mlx4_core
dev.mlx4_core.0.%location: slot=0 function=0 handle=\_SB_.PCI0.BR3C.H000
dev.mlx4_core.0.%pnpinfo: vendor=0x15b3 device=0x1003 subvendor=0x15b3 subdevice=0x1003 class=0x028000
dev.mlx4_core.0.%parent: pci5
sys.device.mlx4_core0.rate_limit_caps.different_rates_num: 0
sys.device.mlx4_core0.mlx4_port1: eth

If anyone can see anything I am missing or doing wrong, I'd be glad to know.

Thanks,
-Fred