Vector Packet Processing (VPP) portability on FreeBSD

[Francois ten Krooden]

Hi

Just for info I ran a test using TREX (https://trex-tgn.cisco.com/)
Where I just sent traffic in one direction through the box running  FreeBSD with VPP using the netmap interfaces.
These were the results we found before significant packet loss started occuring.
+-------------+------------------+
| Packet Size | Throughput (pps) |
+-------------+------------------+
|   64 bytes  |   1.008 Mpps     |
|  128 bytes  |   920.311 kpps   |
|  256 bytes  |   797.789 kpps   |
|  512 bytes  |   706.338 kpps   |
| 1024 bytes  |   621.963 kpps   |
| 1280 bytes  |   569.140 kpps   |
| 1440 bytes  |   547.139 kpps   |
| 1518 bytes  |   524.864 kpps   |
+-------------+------------------+

Still busy investigating as to where this issue originates from.

> -----Original Message-----
> From: Francois ten Krooden
> Sent: Tuesday, 11 May 2021 08:18
> To: 'Luigi Rizzo' <rizzo at iet.unipi.it>
> Cc: freebsd-net at freebsd.org
> Subject: RE: Vector Packet Processing (VPP) portability on FreeBSD
>
> On 2021/05/10 15:39, Luigi Rizzo wrote:
>
> > -----Original Message-----
> > From: Luigi Rizzo [mailto:rizzo at iet.unipi.it]
> > Sent: Monday, 10 May 2021 16:39
> > To: Rainer Duffner <rainer at ultra-secure.de>
> > Cc: Francois ten Krooden <ftk at Nanoteq.com>; freebsd-net at freebsd.org
> > Subject: Re: Vector Packet Processing (VPP) portability on FreeBSD
> >
> > [repost since it appears to be blocked]
> >
> > On Mon, May 10, 2021 at 2:42 PM Rainer Duffner
> > <rainer at ultra-secure.de>
> > wrote:
> > >
> > >
> > >
> > > > Am 10.05.2021 um 13:08 schrieb Francois ten Krooden
> > <ftk at Nanoteq.com>:
> > > >
> > > > Greetings
> > > >
> > > > We have a vested interest in high-speed IPsec VPN on FreeBSD. We
> > > > have
> > started with the porting of VPP (https://fd.io/) to FreeBSD.
> > > >
> > > > Currently we have VPP compiled and running with netmap. The speeds
> > we measure are nowhere near the performance of a 10Gbps link, at
> > around 350kpps for 1500 byte IPv4 packets. We suspect the biggest
> > issue is related to how VPP implements huge pages (Linux) and our
> > modifications to support super pages on FreeBSD.
> >
> > 350kpps is way too low for being related to tlb and page size issues.
> >
> > I suspect that you are
> > either using the "emulated" netmap mode, which runs on top of mbufs
> > and may involve extra data copies on the receive path, or the test itself is
> e.g.
> > using TCP and congestion control or limited window size throttle down
> > the rate.
>
> The NIC's we are using is the 'Intel X552' (10 GbE SFP+) on the Supermicro
> X10SDV-8C-TLN4F+ board.
> I know this uses the 'ixgbe' driver in FreeBSD which is netmap enabled.
> Is there a way I can confirm if it is running in emulated netmap mode?
>
> >
> > I'd retry the test with some open loop traffic source/sink,  and using
> > first all possible low level APIs (sockets/mbufs; emulated netmap;
> > native netmap; your vpp port) with small and large packets and
> > increasing packet rates, to see where the limits are with each. That
> > should give you good hints to figure out what is making the performance
> so bad.
> > Even better if you can run the test between one known-good endpoint so
> > you can test separately the tx and tx sides.
>
> Thanks for these suggestions.  I will rerun the tests again and just check if I
> can make some headway on this.
> One additional test I have done previously is to use 'netmap-fwd' from
> https://github.com/Netgate/netmap-fwd.
> When I executed the tests with netmap-fwd I was able to reach around
> 800kpps for 1500 byte packets, which was almost at the maximum speed.  (I
> will just confirm this number again)
>
> Cheers
> Francois
>
> >
> > Cheers
> > Luigi
> >



Important Notice:

This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail legal notice available at:
http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx