From nobody Wed Sep 22 09:22:03 2021 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 87AD71277107 for ; Wed, 22 Sep 2021 09:22:10 +0000 (UTC) (envelope-from SRS0=+pQ9Po=OM=codenetworks.net=sm@eigbox.net) Received: from bosmailout02.eigbox.net (bosmailout02.eigbox.net [66.96.184.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HDt9F2J6hz4k8t for ; Wed, 22 Sep 2021 09:22:09 +0000 (UTC) (envelope-from SRS0=+pQ9Po=OM=codenetworks.net=sm@eigbox.net) Received: from bosmailscan08.eigbox.net ([10.20.15.8]) by bosmailout02.eigbox.net with esmtp (Exim) id 1mSyS8-0007tW-Kx for freebsd-net@freebsd.org; Wed, 22 Sep 2021 05:22:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codenetworks.net; s=dkim; h=Sender:Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:From:References:To:Subject:Reply-To: Cc:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=yDmMa8UdGwiGUbUn3hXok3ibgoMnpVS5X/Nyo9a9Aos=; b=sGBHTQx5tpTU1XigWKyATpP21j KoWwcREdQyrnmrnpl+TuQwSP7Epq/fZDFLU8ehTC+WyqSucyLTDEwuB0FYCb+W5cYao3bbFMT8FJ/ Z/sMwdJT/GIKrFy4gKYYLIxo4SdouFySkZ5WqG/shuvb6Yr+wFZ1waRXzVpGr9k8M/+hFU3GcDtzt LsajyRZfO6tmYS/Ijy9TjaNqr1Du+p1a1+GppACHMJCsrKTxDSDfx5kERTSpJgQVsyfLJ+k/M1CIh WnM+8upr8fk8ipZdwuZtb8kyydzyYnBG5qqGwBTa9tE2E73Vt+q9yeTzc20r2MEHE8vB8UY9SyvWq 0Ip67hhA==; Received: from [10.115.3.32] (helo=bosimpout12) by bosmailscan08.eigbox.net with esmtp (Exim) id 1mSyS8-000161-8R for freebsd-net@freebsd.org; Wed, 22 Sep 2021 05:22:08 -0400 Received: from bosauthsmtp03.yourhostingaccount.com ([10.20.18.3]) by bosimpout12 with id wxN52500Y03yW7601xN8Dm; Wed, 22 Sep 2021 05:22:08 -0400 X-Authority-Analysis: v=2.3 cv=d4VuNSrE c=1 sm=1 tr=0 a=6uKCkKhFq2wXOH2GoQX8aA==:117 a=0uQB4SeFoYTQnAu9nHJYpg==:17 a=IkcTkHD0fZMA:10 a=7QKq2e-ADPsA:10 a=ZXIg6Xh_i64A:10 a=WBfKHoA4AAAA:8 a=P79nY0tQOE5fhe-CbtcA:9 a=QEXdDO2ut3YA:10 a=6YAGbPlI47EA:10 a=HSBlF4sWv1d7km1GD3FH:22 Received: from cm-81-9-193-231.telecable.es ([81.9.193.231]:42193 helo=[192.168.1.100]) by bosauthsmtp03.eigbox.net with esmtpa (Exim) id 1mSyS4-0003VN-Um for freebsd-net@freebsd.org; Wed, 22 Sep 2021 05:22:05 -0400 Subject: Re: Vector Packet Processing (VPP) portability on FreeBSD To: freebsd-net@freebsd.org References: <20210517192054.0907beea@x23> From: Santiago Martinez Message-ID: <33b2b043-f3b7-e8e0-c290-f754082e0692@codenetworks.net> Date: Wed, 22 Sep 2021 11:22:03 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-EN-UserInfo: d3bdfab0736480cedf04ed92aaea2ef5:931c98230c6409dcc37fa7e93b490c27 X-EN-AuthUser: sm@codenetworks.net X-EN-OrigIP: 81.9.193.231 X-EN-OrigHost: cm-81-9-193-231.telecable.es X-Rspamd-Queue-Id: 4HDt9F2J6hz4k8t X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none ("invalid DKIM record") header.d=codenetworks.net header.s=dkim header.b=sGBHTQx5; dmarc=none; spf=pass (mx1.freebsd.org: domain of "SRS0=@eigbox.net" designates 66.96.184.2 as permitted sender) smtp.mailfrom="SRS0=@eigbox.net" X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.96.128.0/18]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[codenetworks.net: no valid DMARC record]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DKIM_TRACE(0.00)[codenetworks.net:~]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[66.96.184.2:from]; R_DKIM_PERMFAIL(0.00)[codenetworks.net:s=dkim]; FORGED_SENDER(0.30)[sm@codenetworks.net,SRS0=@eigbox.net]; RWL_MAILSPIKE_POSSIBLE(0.00)[66.96.184.2:from]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:29873, ipnet:66.96.128.0/18, country:US]; TAGGED_FROM(0.00)[pQ9Po=OM=codenetworks.net=sm]; FROM_NEQ_ENVFROM(0.00)[sm@codenetworks.net,SRS0=@eigbox.net]; RECEIVED_SPAMHAUS_PBL(0.00)[81.9.193.231:received] X-ThisMailContainsUnwantedMimeParts: N Hi Francois, I hope you are doing well. It is great to hear about work/progress/updates on VPP / DPDK / Netmap on FBSD, even if the results are not the best. Unfortunately, I'm not a developer, so I cannot help much on the matter of the missing bits. Just wondering if those modifications that your team have done to make VPP run can be upstreamed or shared with the community, so maybe we can create a VPP package making it easier for others to deploy/test/improve. On the other hand, do you roughly know how much effort is required to make VFIO support at the same level as Linux? I hope it makes sense. Best regards. Santiago On 9/21/21 11:52 AM, Francois ten Krooden wrote: > Hi > > This is just some feedback for those who had an interest in this topic. > > After spending quite some time on the VPP to FreeBSD porting effort where we did manage to get VPP working with netmap, and VPP compiling with DPDK; We realised that there are some big issues that we would need to overcome. Some of these efforts are not viable for our small team to accomplish in a reasonable time frame. > The main issues that we have found are: > - Tests proved that netmap would not deliver the desired performance as it is currently implemented within VPP. The main issues here are that for every 256 packets memory seems to be allocated again, also a number of copies occur in the memory which slows down the performance. > - VPP relies on VFIO to map device memory into user space for processing within the application. This code is implemented in DPDK in the Linux implementation but in the FreeBSD implementation in DPDK these functions are stubbed. > - To interface with crypto-offloading hardware such as the QAT card from Intel, or our own card VPP/DPDK also utilize VFIO with the PCI device. > - As far as we have been able to see the VFIO support in FreeBSD is not at the same level as Linux, which would then require additional kernel development which is not possible in the time frame. > > Regards > > Francois ten Krooden > Principal Developer > > > Tel: +27 12 672 7000 > Fax: +27 12 665 1343 > Postal: P.O. Box 7991, Centurion, 0046 > Physical: Unit C01, Corporate Park 66, 269 Von Willich Avenue, Centurion > > > > Important Notice: > > This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail legal notice available at: > http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx