netmap: recommended NIC for 40GbE capture on Linux?

[Luigi Rizzo]

On Sat, Oct 10, 2015 at 4:03 PM, Bruce Merry <bmerry at ska.ac.za> wrote:
> Hi
>
> We're aim to do some high-speed data capture and processing on data
> being sent from an FPGA over Ethernet. The data is about 3.3Mpps with
> 512 bytes of application-level data per packet (plus somewhere around
> 64 bytes of app-level headers, plus UDP+IP+Ethernet headers). I'm
> looking for some advice on what hardware to use.
>
> We currently have Mellanox ConnectX-3 NICs with QSFP+. My experiments
> with pkt-gen on a single flow (generated by pkt-gen on another
> machine) are topping out at receiving 2.4 Mpps, with ksortirqd pushing
> one core at 100%. This is with netmap trunk, which does not build a
> custom mlx4 driver. I've also tried
> https://github.com/caldweba/netmap, which does, but which is broken
> (receives only 1023 pps, confirmed broken by the repo owner).
>
> Before we spend money, I'd love to hear someone report success with
> capturing a single flow at >4Mpps, >20Gbps using netmap on Linux and
> what NIC they use. Ideally this would include transparent mode working
> so that ARP, ICMP etc can be handled by the host stack.
>
> Alternatively, if anyone is managing this on ConnectX-3, I'd love to
> know how (I have already disabled flow control and adaptive interrupt
> mitigation on the receiver).
>

hi Bruce,
email me privately for more details.

We have native netmap support in FreeBSD for the intel 40G and
and almost ready linux version of the same, which we hope to be
available by the end of the year.

Recently mellanox kindly sent us a couple of their 40g cards
(not sure which type), for which i plan to adapt the mlx4_netmap_linux
code that you found in the other repo (which i wrote a few years ago).
Again we may work on that around the end of the year.

cheers
luigi

-- 
-----------------------------------------+-------------------------------
 Prof. Luigi RIZZO, rizzo at iet.unipi.it  . Dip. di Ing. dell'Informazione
 http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
 TEL      +39-050-2217533               . via Diotisalvi 2
 Mobile   +39-338-6809875               . 56122 PISA (Italy)
-----------------------------------------+-------------------------------