Vector Packet Processing (VPP) portability on FreeBSD

[Francois ten Krooden]

On 2021/05/10 15:39, Luigi Rizzo wrote:

> -----Original Message-----
> From: Luigi Rizzo [mailto:rizzo at iet.unipi.it]
> Sent: Monday, 10 May 2021 16:39
> To: Rainer Duffner <rainer at ultra-secure.de>
> Cc: Francois ten Krooden <ftk at Nanoteq.com>; freebsd-net at freebsd.org
> Subject: Re: Vector Packet Processing (VPP) portability on FreeBSD
>
> [repost since it appears to be blocked]
>
> On Mon, May 10, 2021 at 2:42 PM Rainer Duffner <rainer at ultra-secure.de>
> wrote:
> >
> >
> >
> > > Am 10.05.2021 um 13:08 schrieb Francois ten Krooden
> <ftk at Nanoteq.com>:
> > >
> > > Greetings
> > >
> > > We have a vested interest in high-speed IPsec VPN on FreeBSD. We have
> started with the porting of VPP (https://fd.io/) to FreeBSD.
> > >
> > > Currently we have VPP compiled and running with netmap. The speeds
> we measure are nowhere near the performance of a 10Gbps link, at around
> 350kpps for 1500 byte IPv4 packets. We suspect the biggest issue is related
> to how VPP implements huge pages (Linux) and our modifications to support
> super pages on FreeBSD.
>
> 350kpps is way too low for being related to tlb and page size issues.
>
> I suspect that you are
> either using the "emulated" netmap mode, which runs on top of mbufs and
> may involve extra data copies on the receive path, or the test itself is e.g.
> using TCP and congestion control or limited window size throttle down the
> rate.

The NIC's we are using is the 'Intel X552' (10 GbE SFP+) on the Supermicro X10SDV-8C-TLN4F+ board.
I know this uses the 'ixgbe' driver in FreeBSD which is netmap enabled.
Is there a way I can confirm if it is running in emulated netmap mode?

>
> I'd retry the test with some open loop traffic source/sink,  and using first all
> possible low level APIs (sockets/mbufs; emulated netmap; native netmap;
> your vpp port) with small and large packets and increasing packet rates, to
> see where the limits are with each. That should give you good hints to figure
> out what is making the performance so bad.
> Even better if you can run the test between one known-good endpoint so you
> can test separately the tx and tx sides.

Thanks for these suggestions.  I will rerun the tests again and just check if I can make some headway on this.
One additional test I have done previously is to use 'netmap-fwd' from https://github.com/Netgate/netmap-fwd.
When I executed the tests with netmap-fwd I was able to reach around 800kpps for 1500 byte packets, which was almost at the maximum speed.  (I will just confirm this number again)

Cheers
Francois

>
> Cheers
> Luigi
>



Important Notice:

This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail legal notice available at:
http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx