OFED setup and Mellanox cards

[Anthony Cornehl]

The /sys folder is abstracted to live in the sysctls. There is some
abstraction in the Linux-compat layer, in /usr/src/sys/ofed/include, that
can allow the use of folder and sysctl in your program from Linux to
FreeBSD.

I'll get a test setup this week.
On Feb 11, 2013 1:29 PM, "Tamer Abuelata" <tamer at polycoresoftware.com>
wrote:

> Hi,
>
> I'm trying to get OFED up and running on a FreeBSD 9.1 machine with 2
> Mellanox cards (MT26428).
> We have compiled the Kernel with the following options (per
> https://wiki.freebsd.org/InfiniBand ):
>
> options        OFED            # Infiniband protocol stack and support
> options        SDP              # Sockets Direct Protocol for infiniband
> options        IPOIB_CM    # Use connect mode ipoib
>
> device         ipoib             # IP over IB devices
> device         mlx4ib          # ConnectX Infiniband support
> device         mlxen           # ConnectX Ethernet support
> device         mthca           # Infinihost cards
>
> I gave an IP to each card.
> # ifconfig
> ib0: flags=8043<UP,BROADCAST,RUNNING,MULTICAST> metric 0 mtu 65520
>         options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
>         lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.25.90.ff.ff.7.70.d9
>         inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
>         nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> ib1: flags=8043<UP,BROADCAST,RUNNING,MULTICAST> metric 0 mtu 65520
>         options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
>         lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.25.90.ff.ff.7.70.f1
>         inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255
>         nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>
> After doing a build world and build kernel I try to run an example
> application and get the following warnings/errors:
>
> # ./server
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs1
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs0
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs1
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs0
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs1
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs0
> listening on port 29823.
>
> Then I launch the client from another terminal:
> # ./client 10.0.0.2 29823
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs1
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs0
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs1
> libibverbs: Warning: no userspace device-specific driver found for
> /sys/class/infiniband_verbs/uverbs0
> [… shows several more times …]
> Segmentation fault (core dumped)
>
> So the sample app compiles fine but I get this runtime error which seems
> to come from the libibverbs. This /sys/class/infiniband_verbs folder does
> not exist. I'm not sure if I'm missing configuration files. However I have
> devices at /dev/uverbs0 and /dev/uverbs1.
>
> The example app I am using is from:
>
> http://thegeekinthecorner.wordpress.com/2010/08/14/building-an-rdma-capable-application-with-ib-verbs-part-2-the-server/
>
> The code is linked at the bottom of the page. Direct link:
>
> https://sites.google.com/a/bedeir.com/home/basic-rdma-client-server.tar.gz?attredirects=0&d=1
>
> I would appreciate any help or pointers from someone who managed to get
> OFED to run on FreeBSD.
>
> Best regards,
> Tamer
> _______________________________________________
> freebsd-infiniband at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-infiniband
> To unsubscribe, send any mail to "
> freebsd-infiniband-unsubscribe at freebsd.org"
>