From nobody Wed Jul 26 09:33:23 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R9pcL2Thbz4nlxT for ; Wed, 26 Jul 2023 09:33:38 +0000 (UTC) (envelope-from marius@btk.zeist.de) Received: from btk.zeist.de (btk.zeist.de [IPv6:2a01:4f8:221:2f4f::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "btk.zeist.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R9pcK710tz3HZ0 for ; Wed, 26 Jul 2023 09:33:37 +0000 (UTC) (envelope-from marius@btk.zeist.de) Authentication-Results: mx1.freebsd.org; none Received: from btk.zeist.de (localhost [127.0.0.1]) by btk.zeist.de (8.17.1/8.17.1/BTK.ZEIST.DE) with ESMTPS id 36Q9XOqq042416 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 26 Jul 2023 11:33:24 +0200 (CEST) (envelope-from marius@btk.zeist.de) Received: (from marius@localhost) by btk.zeist.de (8.17.1/8.17.1/Submit) id 36Q9XNUU042415; Wed, 26 Jul 2023 11:33:23 +0200 (CEST) (envelope-from marius) Date: Wed, 26 Jul 2023 11:33:23 +0200 From: Marius Strobl To: Kevin Bowling Cc: FreeBSD Net Subject: Re: CFT: lem(4), em(4) e1000 Ethernet TSO testing Message-ID: References: List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4R9pcK710tz3HZ0 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated On Tue, Jul 25, 2023 at 07:38:29PM -0700, Kevin Bowling wrote: > Hi, > > I have been working through various bugs and have come to a point > where TSO is working on systems I have available for testing. > > This results in higher throughput on resource constrained systems, and > less CPU/power usage on unconstrained systems. > > As of this mail, you will need to manually apply > https://reviews.freebsd.org/D41170 on top of main to use TSO6 on > em(4). > > I plan to enable TSO by default for lem(4) and em(4) during the > FreeBSD 14 release cycle, so I would appreciate testing to address any > remaining issues. Below, a list of chipsets that will be exempt due > to known issues. > > lem(4) exclusions: > * <82544 (although it does seem ok to manually enable for emulations > in qemu, virtualbox, etc) > * 82547 > > em(4) exclusions.. These chips have a stability workaround for high > throughput with rapid link-flap applied that results in the TSO engine > not being able to run at line speed. Thus, TSO would not be enabled > by default here: > * Intel(R) I219-LM and I219-V > * Intel(R) I219-LM and I219-V (2) > * Intel(R) I219-LM and I219-V (3) > * Intel(R) I219-LM and I219-V (4) > * Intel(R) I219-LM and I219-V (5) In the past, at least for 82579 TSO4 didn't work at all, causing MAC hangs. For devices like 82573E in turn TSO4 only worked at GbE speed. Thus, b1b6afa587761c5d56b0a5410a2a8ea9c2248a57 would automatically disable TSO4 based on actual link speed if needed but that code was lost with the conversion to iflib(9). Thus, given the amount of chips supported, uncertainty whether all required workarounds are actually in place, oddities such as lower link speeds causing problems for TSO, the general mediocre benefit of TSO at GbE and below and the fact that you only have tested TSO with a limited amount of real hardware accroding to D41170, I still don't think it's a good idea to enable TSO by default for EM- and LEM-class gear, especially _not_ during a release cycle. At most, TSO should be whitelisted for specific, thoroughly tested PCI IDs but not the entire classes. Marius