From nobody Thu Nov 10 01:15:02 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N73mY50rSz4XFK3 for ; Thu, 10 Nov 2022 01:15:25 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4N73mY1hGFz3n3f for ; Thu, 10 Nov 2022 01:15:25 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Authentication-Results: mx1.freebsd.org; none Received: from mail.distal.com (pool-108-45-159-88.washdc.fios.verizon.net [108.45.159.88]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Thu, 10 Nov 2022 01:15:04 +0000 Received: from smtpclient.apple ( [2001:470:e24c:200:a1d7:e65e:7dcc:cbfe]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id 9e1f6e44 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Wed, 9 Nov 2022 20:15:02 -0500 (EST) Content-Type: text/plain; charset=utf-8 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.60.0.1.1\)) Subject: Re: Lagg and multi-gigabit questions/proablems From: Chris Ross In-Reply-To: <4093BDE8-D541-4D33-B40C-DA468B0A17B2@punkt.de> Date: Wed, 9 Nov 2022 20:15:02 -0500 Cc: freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <26184FD6-9497-4EE8-B3AC-10C93CE4FFC3@distal.com> References: <76E0E023-AB86-453A-B95C-AED44E7ADA2D@distal.com> <4093BDE8-D541-4D33-B40C-DA468B0A17B2@punkt.de> To: "Patrick M. Hausen" X-Mailer: Apple Mail (2.3693.60.0.1.1) X-Rspamd-Queue-Id: 4N73mY1hGFz3n3f X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[freebsd]; ASN(0.00)[asn:3842, ipnet:168.235.92.0/22, country:US] X-ThisMailContainsUnwantedMimeParts: N > On Nov 9, 2022, at 14:32, Patrick M. Hausen wrote: >=20 >> Am 08.11.2022 um 06:38 schrieb Chris Ross : >> I have a newer Freebsd 12.3 system with lagg across two 1gbe = interfaces. There are a collection of vlan interfaces on the lagg. >>=20 >> I would _expect_ to be able to get 2gbps, or just shy thereof. >=20 > You are aware that LAGG/LACP will give you only a single interface's = bandwidth > for a single stream? That's by design, because reordering of TCP = packets is bad. >=20 > You would need to test with multiple endpoints, because most commonly = distribution > is per IP address hash. Ahh, thank you. While that makes perfect sense and I may have known it = at some point, I didn=E2=80=99t realize that now. I was of course testing = with only the=20 one stream. Just now, I ran two iperf3 streams at the same time, from the same = hosts. For control, from my Mac with Gbe to the server, which got ~950 Mbits/s in a single stream, got a consistent ~430 on each of two streams. =46rom the FreeBSD system with 2Gbe LAGG, my results were not as consistent, but above 1Gbit/s. Sometimes it will run at 950 Mbits/s and 700 Mbits/s in parallel, but other times it will run at variable = speeds more like 600-700 each. In all cases, I think the total was 1200 Mbits/s or more. I=E2=80=99m curious why I was seeing some stream at consistent high = speeds, and others at varying lower speeds, but I=E2=80=99ll just write that off = as load or activity on the hosts. Thanks all. Sorry for the noise, hopefully this being in the archives will help someone else. :-) - Chris=