Vector Packet Processing (VPP) portability on FreeBSD

[Francois ten Krooden]

On  Monday, 10 May 2021 16:10 Konstantin Belousov wrote:


> On Mon, May 10, 2021 at 11:08:18AM +0000, Francois ten Krooden wrote:
> > Greetings
> >
> > We have a vested interest in high-speed IPsec VPN on FreeBSD. We have
> started with the porting of VPP (https://fd.io/) to FreeBSD.
> >
> > Currently we have VPP compiled and running with netmap. The speeds we
> measure are nowhere near the performance of a 10Gbps link, at around
> 350kpps for 1500 byte IPv4 packets. We suspect the biggest issue is related
> to how VPP implements huge pages (Linux) and our modifications to support
> super pages on FreeBSD.
> >
> > Apart from the above, there are remaining issues we need to sort out and
> "Linuxisms" that need porting to FreeBSD, but this is going reasonably well.
> We are working in a public Github repository and have started listing our
> issues there alongside the code. Our main working branch is "freebsd"
> (https://github.com/ftk-ntq/vpp/tree/freebsd).
> >
> > Our aim with this mail is to get the discussion started on porting VPP to
> FreeBSD and to invite interested parties to help with the effort. We intend
> to upstream the work hoping that the original authors will adopt our ported
> code and continue maintaining future compatibility with FreeBSD.
> >
> > Some of our questions or comments to start the conversation:
> > 1. netmap vs. DPDK (VPP relies on DPDK by default with the netmap
> integration deprecated). Which will be the best to choose?
> > 2. How to correctly implement using super pages / huge pages in FreeBSD
> in order to allow VPP to allocate contiguous memory blocks for packet
> buffers to process packets from the packet handling framework
> (netmap/DPDK)?
> Superpage CPU mappings, as opposed to small pages mappings, typically
> give you several units of percents improvement in best setup. Having large
> page support in hardware for things like memory registration/rings mapping
> could give you some more if hardware DMA engine is limited in capacity,
> e.g. have fixed number of descriptors.
>
> To allocate physically-contiguous superpage-mapped regions in userspace,
> create special posix shmfd with shm_create_largepage(), then map it with
> mmap(2).  The backing pages are allocated on creation, so you can e.g. pass
> them to hardware without mmaping into userspace at all, if
> needed/preferred.

Thank you I will have a look at this as well.  I knew about memfd_create which was also added in FreeBSD 13.0 which I did use.


>
> > 3. What are suitable alternatives for reading information from procfs and
> sysfs on FreeBSD?
> Understand what information is obtained, then what for is it actually used,
> then match it against equivalent FreeBSD approach, then gather the
> required information.

Thank you.  This was basically what we suspected.
One of the ones we are unsure about is what the equivalent of /proc/self/pagemap on Linux would be.
The one idea we had is using procstat_getvmmap from libprocstat, but haven't finished investigating yet.

Regards
Francois

>
> > 4. Functionality relying on Linux epoll is currently supported using epoll-
> shim. Is this the correct approach?
> >
> > Any help and input to aid in the effort will be greatly appreciated.
> >
> > Regards
> >
> > Francois ten Krooden
> >
> >
> > Important Notice:
> >
> > This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail
> legal notice available at:
> > http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx
> >
> >
> > _______________________________________________
> > freebsd-net at freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>



Important Notice:

This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail legal notice available at:
http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx