Vector Packet Processing (VPP) portability on FreeBSD

[Francois ten Krooden]

On Thursday, 13 May 2021 13:59 Jacques Fourie

>
> On Thu, May 13, 2021 at 7:27 AM Francois ten Krooden <ftk at nanoteq.com>
> wrote:
>
> On Thursday, 13 May 2021 13:05 Luigi Rizzo wrote:
> >
> > On Thu, May 13, 2021 at 10:42 AM Francois ten Krooden
> > <ftk at nanoteq.com> wrote:
> > >
> > > Hi
> > >
> > > Just for info I ran a test using TREX (https://trex-tgn.cisco.com/)
> > > Where I just sent traffic in one direction through the box
> running  FreeBSD
> > with VPP using the netmap interfaces.
> > > These were the results we found before significant packet loss started
> > occuring.
> > > +-------------+------------------+
> > > | Packet Size | Throughput (pps) |
> > > +-------------+------------------+
> > > |   64 bytes  |   1.008 Mpps     |
> > > |  128 bytes  |   920.311 kpps   |
> > > |  256 bytes  |   797.789 kpps   |
> > > |  512 bytes  |   706.338 kpps   |
> > > | 1024 bytes  |   621.963 kpps   |
> > > | 1280 bytes  |   569.140 kpps   |
> > > | 1440 bytes  |   547.139 kpps   |
> > > | 1518 bytes  |   524.864 kpps   |
> > > +-------------+------------------+
> >
> > Those numbers are way too low for netmap.
> >
> > I believe you are either using the emulated mode, or issuing a system call
> on
> > every single packet.
> >
> > I am not up to date (Vincenzo may know better) but there used to be a
> sysctl
> > variable to control the operating mode:
> >
> > https://www.freebsd.org/cgi/man.cgi?query=netmap&sektion=4
> >
> > SYSCTL VARIABLES AND MODULE PARAMETERS
> >      Some aspects of the operation of netmap and VALE are controlled
> > through
> >      sysctl variables on FreeBSD (dev.netmap.*) and module parameters on
> > Linux
> >      (/sys/module/netmap/parameters/*):
> >
> >      dev.netmap.admode: 0
> >      Controls the use of native or emulated adapter mode.
> >
> >      0 uses the best available option;
> >
> >      1 forces native mode and fails if not available;
> >
> >      2 forces emulated hence never fails.
> >
> > If it still exists, try set it to 1. If the program fails, then you should figure
> out
> > why native netmap support is not compiled in.
>
> Thank you.  I did set this to 1 specifically now and it still works.  So then it
> should be running in native mode.
>
> I will dig a bit into the function that processes the incoming packets.
> The code I currently use was added to VPP in somewhere before 2016, so it
> might be that there is a bug in that code.
>
> Will try and see if I can find anything interesting there.
>
> >
> > cheers
> > luigi
> >
> A couple of questions / suggestions:

Thank you for the suggestions.

> Will it be possible to test using the netmap bridge app or a vale switch
> instead of vpp?
I did perform a test using netmap-fwd (https://github.com/Netgate/netmap-fwd)
I did look at the code and it appears that the packets are processed as a batch in the application.  But each packet is passed through the complete IP stack in the application, before the next one is processed.
With this application it was possible to reach about 1.4Mpps for 64-byte packets, and 812 kpps for 1518 byte packets
I haven't done any other tweaking on the FreeBSD box yet.  It is running FreeBSD 13.0

> Did you verify that the TREX setup can perform at line rate when connected
> back to back?
We did tests with TREX back to back yesterday and we reached the following.
+-------------+------------------+
| Packet Size | Throughput (pps) |
+-------------+------------------+
|   64 bytes  |    14.570 Mpps   |
|  128 bytes  |     8.466 kpps   |
|  256 bytes  |     4.542 kpps   |
|  512 bytes  |     2.354 kpps   |
| 1024 bytes  |     1.200 kpps   |
| 1280 bytes  |   965.042 kpps   |
| 1440 bytes  |   857.795 kpps   |
| 1518 bytes  |   814.690 kpps   |
+-------------+------------------+

> Which NICs are you using?
We are using Intel X552 10 GbE SFP+ NIC's which is part of the Intel Xeon D-1537 SoC, on a SuperMicro X10SDV-8C-TLN4F+ Board.

I will also put the results on the github repository
https://github.com/ftk-ntq/vpp/wiki
and will update as we get some more information

Kind Regards
Francois

>
>
> Important Notice:
>
> This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail legal
> notice available at:
> http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"


Important Notice:

This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail legal notice available at:
http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx