netmap: recommended NIC for 40GbE capture on Linux?

[Bruce Merry]

Hi

We're aim to do some high-speed data capture and processing on data
being sent from an FPGA over Ethernet. The data is about 3.3Mpps with
512 bytes of application-level data per packet (plus somewhere around
64 bytes of app-level headers, plus UDP+IP+Ethernet headers). I'm
looking for some advice on what hardware to use.

We currently have Mellanox ConnectX-3 NICs with QSFP+. My experiments
with pkt-gen on a single flow (generated by pkt-gen on another
machine) are topping out at receiving 2.4 Mpps, with ksortirqd pushing
one core at 100%. This is with netmap trunk, which does not build a
custom mlx4 driver. I've also tried
https://github.com/caldweba/netmap, which does, but which is broken
(receives only 1023 pps, confirmed broken by the repo owner).

Before we spend money, I'd love to hear someone report success with
capturing a single flow at >4Mpps, >20Gbps using netmap on Linux and
what NIC they use. Ideally this would include transparent mode working
so that ARP, ICMP etc can be handled by the host stack.

Alternatively, if anyone is managing this on ConnectX-3, I'd love to
know how (I have already disabled flow control and adaptive interrupt
mitigation on the receiver).

Thanks
Bruce
-- 
Bruce Merry
Senior Science Processing Developer
SKA South Africa