From nobody Tue May 23 21:23:46 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QQnPM64tmz4T411 for ; Tue, 23 May 2023 21:23:51 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QQnPM58Pgz3ymc; Tue, 23 May 2023 21:23:51 +0000 (UTC) (envelope-from kp@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684877031; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K+kfWaU5Yu3OCnvrdjWniKjQIOpoqxItysvQQudLKCM=; b=BEB51nJ6iLYUmeMadEZpuVRr9oom6Kykzpa9stoTXJHoH8owgr1Pn1g85WOt7y2nwYHTu7 fW5yq37+ATqzhe0/6nvMfU6tFMA0GVKcr0aKl3md98/3Gpq+ZVveCAXQKU0lMO0EwrRWSF 3j5tmfVEv9P7jHKLp9kTydQHkBL529RJGBcO/ysxllYXO2TJyF8gFSqknPxx5Zdb1NuasG DosOcAfHtGO0WAdz1BEuqc8tQfrwvTJsybYydrrig/aISIY3SY1teJOfwDVZH7u2hN+m8l 7IeyI7W90ETiHjtKeN6L6lxXfRo1ikeVTT7Qdw+Hetg4PZ/EU1hSvwL4ZUkGTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684877031; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K+kfWaU5Yu3OCnvrdjWniKjQIOpoqxItysvQQudLKCM=; b=H3k07pR9ZEaKFfM+hbCTLnC2wf5zf/sHs/t1MDK0tXerl4p3E6zglUZvltCJ4wH96xkXSw BTtRwsdAhPmLGUroU8N8RIJAdTykpOX17sbQ4PnMlSHv9+hYt7Fir4srZoHWH4QGrHXCe6 oeuxgrSxG6GmLXaiAHLfTd7msy7CixX9O/5qTOJ8dunvpb/+21mLkwj2HivuxE83nQutGS TxiBtQchc9i7yuChvGYTNuIN97hq78uGz71mezt0WAllntV4AM5ci9gSTyx8SLTAS3ehNj T6+qbu//f7+zkrcJ9iwdjxguspgsMFflya/x1SkGHvMdujheTNvIexjOnIWLTQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1684877031; a=rsa-sha256; cv=none; b=DsaaAv+/YseqjpFNs1Avvys27Mxb9PuXtfsCKfSS4GVX8tAwzG8QYvXbY+8Tza1CpEg0LG cXIvOOhM53jD2KbgZw/5E7cPd3yUxssfKqsJf+04WMH+MHULOV3Rnt8BbEUeyV/DaxAkwa duIsG8mFCel47Qc+DU0CN4pPsbe+urdoFicUbC5SynhBIJ4rQpg1sRlBPiMVG5zNWYmKAa G7rELzO5tmKaBtx/BR5ufeXdHxcb+8/ylFxK/vjVtbcmn0750N8nnkWHS7ACKOjRo62e3S tf9Szj3z+v9tog853ZAKUXiuaw5AXrXncQtZ1ue98p3pDPm4j4K4mLb+kpOlmg== Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 4QQnPM3V5Kzh0g; Tue, 23 May 2023 21:23:51 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id 90606263DA; Tue, 23 May 2023 23:23:49 +0200 (CEST) From: Kristof Provost To: Benoit Chesneau Cc: freebsd-net@freebsd.org Subject: Re: how to increase the vnet speed? Date: Tue, 23 May 2023 16:23:46 -0500 X-Mailer: MailMate (1.14r5937) Message-ID: <84F8F5E0-FF88-4C42-A6E8-7C5F2F458BEF@FreeBSD.org> In-Reply-To: References: List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-ThisMailContainsUnwantedMimeParts: N On 23 May 2023, at 14:58, Benoit Chesneau wrote: > I've created a jail using bastille and setup network. The mainin terfac= e is a 25Gbps nic and between hosts I get 24.6 Gbits/sec : > > [ ID] Interval Transfer Bitrate[ 5] 0.00-1.00 sec 2.87 GBytes 24.6 Gbit= s/sec > [ 5] 1.00-2.00 sec 2.88 GBytes 24.7 Gbits/sec > [ 5] 2.00-3.00 sec 2.87 GBytes 24.7 Gbits/sec > [ 5] 3.00-4.00 sec 2.87 GBytes 24.7 Gbits/sec > [ 5] 4.00-5.00 sec 2.87 GBytes 24.7 Gbits/sec > [ 5] 5.00-6.00 sec 2.87 GBytes 24.7 Gbits/sec > [ 5] 6.00-7.00 sec 2.87 GBytes 24.7 Gbits/sec > [ 5] 7.00-8.00 sec 2.87 GBytes 24.7 Gbits/sec > [ 5] 8.00-9.00 sec 2.87 GBytes 24.7 Gbits/sec > [ 5] 9.00-10.00 sec 2.87 GBytes 24.7 Gbits/sec > [ 5] 10.00-10.00 sec 128 KBytes 17.2 Gbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate > [ 5] 0.00-10.00 sec 28.7 GBytes 24.7 Gbits/sec receiver > > But between one host and the jail I only get 3.96 Gbits/sec > > ID] Interval Transfer Bitrate Retr Cwnd[ 5] 0.00-1.00 sec 477 MBytes 4.= 00 Gbits/sec 44 278 KBytes > [ 5] 1.00-2.00 sec 455 MBytes 3.82 Gbits/sec 41 215 KBytes > [ 5] 2.00-3.00 sec 463 MBytes 3.89 Gbits/sec 61 391 KBytes > [ 5] 3.00-4.00 sec 488 MBytes 4.10 Gbits/sec 39 336 KBytes > [ 5] 4.00-5.00 sec 484 MBytes 4.06 Gbits/sec 30 253 KBytes > [ 5] 5.00-6.00 sec 448 MBytes 3.76 Gbits/sec 44 355 KBytes > [ 5] 6.00-7.00 sec 454 MBytes 3.81 Gbits/sec 56 266 KBytes > [ 5] 7.00-8.00 sec 471 MBytes 3.95 Gbits/sec 41 325 KBytes > [ 5] 8.00-9.00 sec 500 MBytes 4.20 Gbits/sec 39 321 KBytes > [ 5] 9.00-10.00 sec 475 MBytes 3.98 Gbits/sec 42 367 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 4.61 GBytes 3.96 Gbits/sec 437 sender > [ 5] 0.00-10.00 sec 4.61 GBytes 3.96 Gbits/sec receiver > > Is there a way to increase the performance of the of the jail? The nice= is a mellannox ConnectX-4 Lx, mce(4) . > What=E2=80=99s your setup? The source of the bottleneck would be completely different if you route o= r bridge, pass the interface into the vnet or use epair or =E2=80=A6 or .= =2E or. We can=E2=80=99t just guess. Kristof