OFED v3.2 update

[Jason Bacon]

This is encouraging...

On our CentOS 6.7 cluster, what we're seeing is comparable to the Ubuntu 
performance you reported earlier.  ( Best of 3 runs between two idle 
compute nodes on the same leaf switch shown below. )

It appears that we may be able to improve NFS performance using a 
FreeBSD/FreeNAS file server in the near future.

Regards,

     Jason

Linux compute-083. bacon ~ 335: iperf -c compute-082-ib0
------------------------------------------------------------
Client connecting to compute-082-ib0, TCP port 5001
TCP window size:  645 KByte (default)
------------------------------------------------------------
[  3] local 10.2.11.109 port 47470 connected with 10.2.11.108 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  22.3 GBytes  19.2 Gbits/sec

On 04/01/16 13:08, Chris Reeves wrote:
> Thank you Hans,
>
> Using these patches, I was able to get my IPOIB performance up to 25 Gb/s (vs 10 Gb/s on 11-current  without the patches).
>
> Out of curiosity, do you have any plans to MFC these to stable-10?
>
> Thanks,
> Chris
> ________________________________________
> From: owner-freebsd-infiniband at freebsd.org <owner-freebsd-infiniband at freebsd.org> on behalf of Hans Petter Selasky <hps at selasky.org>
> Sent: Thursday, March 31, 2016 3:51 AM
> To: freebsd-infiniband at freebsd.org
> Subject: OFED v3.2 update
>
> Dear users of infiniband in FreeBSD:
>
>
> Mellanox is working on a big infiniband update towards mellanox OFED
> v3.2 of the infiniband stack (ibcore), drivers (mlx4/mlx5) and libraries
> (librdma, libibverbs ...). Currently there are 10-patches in different
> areas of the FreeBSD kernel and userspace which you need to apply to
> 11-current as of today to get a working GENERIC kernel.
>
> Compile world and kernel like this after applying the patches below:
>
> make buildkernel WITH_OFED=YES
> make buildworld WITH_OFED=YES
>
>
> https://reviews.freebsd.org/D5790
> https://reviews.freebsd.org/D5791
> https://reviews.freebsd.org/D5792
> https://reviews.freebsd.org/D5793
> https://reviews.freebsd.org/D5794
> https://reviews.freebsd.org/D5795
> https://reviews.freebsd.org/D5796
> https://reviews.freebsd.org/D5797
> https://reviews.freebsd.org/D5798
> https://reviews.freebsd.org/D5799
>
>
> The patches are work in progress, though at a level where basic
> infiniband is working with MLX5.
>
>
> Known issues:
> - LINT build is broken (Will be fixed before committing. Waiting for
> solution from Navdeep)
> - Need to unload ipoib before mlx5ib.
>
> --HPS
> _______________________________________________
> freebsd-infiniband at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband
> To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe at freebsd.org"
> _______________________________________________
> freebsd-infiniband at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband
> To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe at freebsd.org"


-- 

-------------------------------------
   Jason W. Bacon
   Systems Manager
   Research Computing Support
   University of Wisconsin Milwaukee
   bacon at uwm.edu
-------------------------------------