From nobody Mon Oct 04 11:33:43 2021 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 918E617ADE95 for ; Mon, 4 Oct 2021 11:34:03 +0000 (UTC) (envelope-from ftk@nanoteq.com) Received: from delivery.e-purifier.com (delivery.e-purifier.com [41.168.2.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HNJWt0h52z4dGw for ; Mon, 4 Oct 2021 11:34:02 +0000 (UTC) (envelope-from ftk@nanoteq.com) Received: from sec-ngp-ag01.neotel.e-purifier.co.za ([192.168.202.31] helo=SEC-NGP-AG01) by delivery.e-purifier.com with smtp (Exim 4.94) (envelope-from ) id 1mXMEC-0004YK-H1 for freebsd-net@freebsd.org; Mon, 04 Oct 2021 13:33:52 +0200 Received: from sec-ngp-spt06.e-purifier.com ([192.168.201.1]) by SEC-NGP-AG01.neotel.e-purifier.co.za with Microsoft SMTPSVC(7.5.7601.17514); Mon, 4 Oct 2021 13:33:50 +0200 Received: from localhost (localhost [127.0.0.1]) by sec-ngp-spt06.e-purifier.com (Postfix) with ESMTP id 9570C698249; Mon, 4 Oct 2021 13:33:51 +0200 (SAST) X-Virus-Scanned: by SpamTitan at e-purifier.com Received: from sec-ngp-spt06.e-purifier.com (localhost [127.0.0.1]) by sec-ngp-spt06.e-purifier.com (Postfix) with ESMTP id 57B1A698244; Mon, 4 Oct 2021 13:33:45 +0200 (SAST) Received: from NTQ-EXC.nanoteq.co.za (unknown [41.170.5.18]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sec-ngp-spt06.e-purifier.com (Postfix) with ESMTPS id 4D281698252; Mon, 4 Oct 2021 13:33:45 +0200 (SAST) Received: from NTQ-EXC.nanoteq.co.za ([fe80::a5b3:4700:5af3:78b2]) by NTQ-EXC.nanoteq.co.za ([fe80::a5b3:4700:5af3:78b2%12]) with mapi id 14.03.0513.000; Mon, 4 Oct 2021 13:33:44 +0200 From: Francois ten Krooden To: Santiago Martinez , "freebsd-net@freebsd.org" Subject: RE: Vector Packet Processing (VPP) portability on FreeBSD Thread-Topic: Vector Packet Processing (VPP) portability on FreeBSD Thread-Index: AQHXRapQkx/sKwTM3EOdxg1D+Pz3RqrhREohgAAEi1D//+jugIAAL0oAgAQ6h4CAAdzSEIAAXLoAgADt0YCAAOTRgIAKBngAgAAE4QCAu14JoIABaJuAgBMhefo= Date: Mon, 4 Oct 2021 11:33:43 +0000 Message-ID: References: <20210517192054.0907beea@x23> ,<33b2b043-f3b7-e8e0-c290-f754082e0692@codenetworks.net> In-Reply-To: <33b2b043-f3b7-e8e0-c290-f754082e0692@codenetworks.net> Accept-Language: en-US, en-ZA Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 X-OriginalArrivalTime: 04 Oct 2021 11:33:50.0750 (UTC) FILETIME=[B414D3E0:01D7B913] x-archived: yes x-dbused: RGF0YSBTb3VyY2U9MTkyLjE2OC4yMDEuMjc= X-Rspamd-Queue-Id: 4HNJWt0h52z4dGw X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of ftk@nanoteq.com has no SPF policy when checking 41.168.2.24) smtp.mailfrom=ftk@nanoteq.com X-Spamd-Result: default: False [-2.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[Nanoteq.com]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:36937, ipnet:41.168.0.0/17, country:ZA]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_SEVEN(0.00)[7] X-ThisMailContainsUnwantedMimeParts: N Hi Santiago The patches we have made is all available on the github fork we made from t= he VPP repository. It is located at https://github.com/ftk-ntq/vpp/tree/freebsd So anyone who is interested can find it there. To make the VFIO support work I unfortunately have no idea. I am not exactly sure but I think there is some kernel work required and th= en an update to DPDK. I am not sure how much effort that would be. Kind Regards Francois ten Krooden Principal Developer Nanoteq Tel: +27 12 672 7000 Fax: +27 12 665 1343 Postal: P.O. Box 7991, Centurion, 0046 Physical: Unit C01, Corporate Park 66, 269 Von Willich Avenue, Cent= urion ________________________________________ From: Santiago Martinez [sm@codenetworks.net] Sent: Wednesday, September 22, 2021 11:22 AM To: freebsd-net@freebsd.org Subject: Re: Vector Packet Processing (VPP) portability on FreeBSD Hi Francois, I hope you are doing well. It is great to hear about work/progress/updates on VPP / DPDK / Netmap on FBSD, even if the results are not the best. Unfortunately, I'm not a developer, so I cannot help much on the matter of the missing bits. Just wondering if those modifications that your team have done to make VPP run can be upstreamed or shared with the community, so maybe we can create a VPP package making it easier for others to deploy/test/improve. On the other hand, do you roughly know how much effort is required to make VFIO support at the same level as Linux? I hope it makes sense. Best regards. Santiago On 9/21/21 11:52 AM, Francois ten Krooden wrote: > Hi > > This is just some feedback for those who had an interest in this topic. > > After spending quite some time on the VPP to FreeBSD porting effort where= we did manage to get VPP working with netmap, and VPP compiling with DPDK;= We realised that there are some big issues that we would need to overcome.= Some of these efforts are not viable for our small team to accomplish in a= reasonable time frame. > The main issues that we have found are: > - Tests proved that netmap would not deliver the desired performance as i= t is currently implemented within VPP. The main issues here are that for ev= ery 256 packets memory seems to be allocated again, also a number of copies= occur in the memory which slows down the performance. > - VPP relies on VFIO to map device memory into user space for processing = within the application. This code is implemented in DPDK in the Linux imple= mentation but in the FreeBSD implementation in DPDK these functions are stu= bbed. > - To interface with crypto-offloading hardware such as the QAT card from = Intel, or our own card VPP/DPDK also utilize VFIO with the PCI device. > - As far as we have been able to see the VFIO support in FreeBSD is not a= t the same level as Linux, which would then require additional kernel devel= opment which is not possible in the time frame. > > Regards > > Francois ten Krooden > Principal Developer > > > Tel: +27 12 672 7000 > Fax: +27 12 665 1343 > Postal: P.O. Box 7991, Centurion, 0046 > Physical: Unit C01, Corporate Park 66, 269 Von Willich Avenue, C= enturion > > > > Important Notice: > > This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail = legal notice available at: > http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx Important Notice: This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail le= gal notice available at: http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx