netmap: recommended NIC for 40GbE capture on Linux?

[Sean Bruno]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512



On 10/10/15 07:03, Bruce Merry wrote:
> Hi
> 
> We're aim to do some high-speed data capture and processing on
> data being sent from an FPGA over Ethernet. The data is about
> 3.3Mpps with 512 bytes of application-level data per packet (plus
> somewhere around 64 bytes of app-level headers, plus
> UDP+IP+Ethernet headers). I'm looking for some advice on what
> hardware to use.
> 
> We currently have Mellanox ConnectX-3 NICs with QSFP+. My
> experiments with pkt-gen on a single flow (generated by pkt-gen on
> another machine) are topping out at receiving 2.4 Mpps, with
> ksortirqd pushing one core at 100%. This is with netmap trunk,
> which does not build a custom mlx4 driver. I've also tried 
> https://github.com/caldweba/netmap, which does, but which is
> broken (receives only 1023 pps, confirmed broken by the repo
> owner).
> 
> Before we spend money, I'd love to hear someone report success
> with capturing a single flow at >4Mpps, >20Gbps using netmap on
> Linux and what NIC they use. Ideally this would include transparent
> mode working so that ARP, ICMP etc can be handled by the host
> stack.
> 
> Alternatively, if anyone is managing this on ConnectX-3, I'd love
> to know how (I have already disabled flow control and adaptive
> interrupt mitigation on the receiver).
> 
> Thanks Bruce
> 


I'm unsure where to ask netmap+linux question to be honest.  So, my
responses won't be very useful to you.  If you feel like doing even
*more* science, FreeBSD does have the same capabilities and supports
the card that you're using.  It might be more work for you to do, but
you might find FreeBSD-current (11.0 test) refreshing enough to
warrant more investigation.

ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/amd64/amd64/ISO-IMAGES/11.0/

sean
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQF8BAEBCgBmBQJWGVMoXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRCQUFENDYzMkU3MTIxREU4RDIwOTk3REQx
MjAxRUZDQTFFNzI3RTY0AAoJEBIB78oecn5kyooH/1bIoaytmkUR73ysuRoBTKcY
UHSc7bvngiMLK15cAKRn9UzXoQHjZRQBMXb9mr1omPwdkzsmfx3JAdOIuNhKi0li
nYYth8/Cs/GUyKxjaYBs/oLydKoZahgiaeNaEqftQmxkOINOXMI6xLxr4p1kMy6k
AbF/LkaSgoyBH/Yu96EK7dQBNCroJuI6BKI7EZdLL93seecsTLx78+qEfXJJI7FC
w5vSD5amFMDKMiAewMzhiHzY+uO5rcYzxGbCineF+6aaKT4jbxJ/X6OoEvw8Zzto
tkSmDiwxFG14YAsn0RQ4BXF3NDFQOLcH0YzTe+mSNwkrv5rSmAdUaTADuiBd+2o=
=Sz+N
-----END PGP SIGNATURE-----