From nobody Sun Sep 26 02:59:23 2021 X-Original-To: net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 194DC17E5CC0 for ; Sun, 26 Sep 2021 02:59:42 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-yb1-xb33.google.com (mail-yb1-xb33.google.com [IPv6:2607:f8b0:4864:20::b33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HH9V602Btz4d1D for ; Sun, 26 Sep 2021 02:59:42 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-yb1-xb33.google.com with SMTP id z5so15169596ybj.2 for ; Sat, 25 Sep 2021 19:59:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=PTSoLfPsswrdlX+cLUPvA9dfZZKG+/s3lH10RB55Zw4=; b=hWEgUeOYnuy6ORD8zgpP2TOCG4p/RiOY8FKu/jibJ6nzSFclMOQuBXHDGvoyfzed5Y PDeIjObTMr05ZNfw1k4Uozo79KKZpwTQjQ1SGdK/T/+u6xErme3SmWxgLHlqb3zx2CjR crK6MjB6HY+rzo6mF/H7VL7xNR237Rw//Yvb8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=PTSoLfPsswrdlX+cLUPvA9dfZZKG+/s3lH10RB55Zw4=; b=qCnPB++GCcv5j9i3VvJfI9HZ7z4IM9Miseij9kWO4o/aaS+GW06JMWqJxYT3USOEXZ ly6vf15h6uVjnNt02D1wGC1XVhg/3QxUUiiqpAszHkT98nFWy6m7E1M9ApoQM63FP+L0 TVJSAWl/8bsRv+b7PlGJ/NrFtUy2IBF5Cf7pBab2mzFRWgB3YsEC8MP6Ct4PbLDJEsyg AeO/FJQo1Kdp5mSCxqW+GFps+LIdPDuJqkS5Xj4loAzclruvuhSCb+UfXB2BVa+rrzYS Meka98l8P5CaS/cIBb0M2fm7K24FyN9gOH5zzfA0PkqVHv24+wK0xYFuvacaD5pqYBMl QfQA== X-Gm-Message-State: AOAM530uTktMAr2C5E/9b/glYFDmsWXFJWlzEJw1gH8txm6FYoJ4uF80 y0xuXSHNv5jMekU0VER0kart1DUrwQ3XqKbhU8lGDQ== X-Google-Smtp-Source: ABdhPJyEPDMoVLJOmmRiDvuKn5DfhYWXn99Q9sUqf0A6vH14ByDN61I0eH8DaFOFqOGrl8daKLsltNkaMmCtJI/vaA4= X-Received: by 2002:a5b:cd0:: with SMTP id e16mr35958ybr.533.1632625174903; Sat, 25 Sep 2021 19:59:34 -0700 (PDT) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 References: <8e4f78e5-0717-8002-5364-44df5c8d7dad@omnilan.de> <36d9d998-c484-a4f6-6c62-c6ec103aeb33@yandex.ru> <14f7348c-a11f-9ae8-8a4e-77e0333ba478@omnilan.de> <57df3182-a7ec-112c-c8d8-a8faa21a97a8@omnilan.de> In-Reply-To: <57df3182-a7ec-112c-c8d8-a8faa21a97a8@omnilan.de> From: Kevin Bowling Date: Sat, 25 Sep 2021 19:59:23 -0700 Message-ID: Subject: Re: git: 1a72c3d76aea - stable/13 - e1000: always enable PCSD when RSS hashing [Was: TCP6 regression for MTU path on stable/13] To: Harry Schmalzbauer Cc: Kevin Bowling , "Andrey V. Elsukov" , "net@FreeBSD.org" , stephan.dewt@yahoo.co.uk, Franco Fichtner Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4HH9V602Btz4d1D X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sat, Sep 25, 2021 at 5:53 PM Harry Schmalzbauer wro= te: > > Am 13.09.2021 um 13:18 schrieb Harry Schmalzbauer: > > Am 13.09.2021 um 12:37 schrieb Andrey V. Elsukov: > >> 12.09.2021 14:12, Harry Schmalzbauer =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > >>> Will try to further track it down, but in case anybody has an idea, > >>> what > >>> change during the last view months in stable/13 could have caused thi= s > >>> real-world problem regarding resulting TCP6 throughput, I'm happy to > >>> start testing at that point. > >> > >> Hi, > >> > >> Take a look at: > >> > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255749 > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D248005 > >> > >> does the problem described in these PRs is the same as yours? > > > > Hi, thank you very much for your attention! > > Most likely these are unrelated to the regression I'm suffering from, > > because these affect 13-release and earlier. > > Mine arose during the last months. > > And it seems not to be a jumbo frame problem. > : > > Hope to get back to you soon with more info. > > > Since the setup was hard to replicate, it took some time. > Here's the commit, causing the heavy IPv6 performance drop with Intel > Powerville and IPv6: > > > The branch stable/13 has been updated by kbowling (ports committer): > > > > URL: > > https://cgit.FreeBSD.org/src/commit/?id=3D1a72c3d76aeafe4422ff20f81c414= 2efb983b7d7 > > > > commit 1a72c3d76aeafe4422ff20f81c4142efb983b7d7 > > Author: Kevin Bowling > > AuthorDate: 2021-08-16 17:17:34 +0000 > > Commit: Kevin Bowling > > CommitDate: 2021-08-23 16:23:43 +0000 > > > > e1000: always enable PCSD when RSS hashing > > > > To enable RSS hashing in the NIC, the PCSD bit must be set. > > > > By default, this is never set when RXCSUM is disabled - which > > causes problems higher up in the stack. > > > > While here improve the RXCSUM flag assignments when enabling or > > disabling IFCAP_RXCSUM. > > > > See also: > > https://lists.freebsd.org/pipermail/freebsd-current/2020-May/076148.htm= l > > > > Reviewed by: markj, Franco Fichtner , > > Stephan de Wit > > Obtained from: OPNsense > > MFC after: 1 week > > Differential Revision: https://reviews.freebsd.org/D31501 > > Co-authored-by: Stephan de Wit > > Co-authored-by: Franco Fichtner > > > > (cherry picked from commit 69e8e8ea3d4be9da6b5bc904a444b51958128ff5= ) > > : > > Noticed and successfully (double-{a8446d412+f72cdea25}) falsified with > i350 Powerville, device=3D0x1521. > *Reverting git: 1a72c3d76aea against today's stable/13(-f72cdea25-dirty) > sloves the issue, which seems to be IPv6 related only.* > (kernel a8446d412 from 21/09/25 shows issue, reverting this commit > solves it with old kernel too) > > Very brief check against IPv4 on identical paths seems to be unaffected, > but I can't guarantee since v4 isn't in use (where I 1st noticed and > suffer from) and I just did one comparing in order to narrow down > (asymmetric FIB setup regarding inet and inet6). > > What this made complicated: ng_brige(4), mpd5/pppoe,ppt,bhyve are > involved as well (and vlan(4), lagg(4) and vtnet(4), etc.), but it seems > to be just a e1000 driver issue. > There were many changes/iprovements/cleanups between July and September, > but I tracked it down as root cause for my IPv6 issue (performance > dropping from 33MB/s to <=3D0.3MB/s). > > > That beeing said, it was hard to find the time replicating the setup, > and I have nothing for a solution. Haven't semantically checked > anything yet and didn't do any tests beside my single IPv6 performance > test. Contrary to my first suspicion, at least in my clone-lab, it > isn't MTU/jumbo frame related, just plain e1000/i350 IPv6 regression. > > > Happy to test anything, can test-drive swiftly but without further diag > during work days. > > Thanks, > -harry Thanks for the report. I added Franco and Stephen to the cc for visibility= . Nothing is immediately jumping out at me, in a private email Harry tried not setting 'rxcsum |=3D E1000_RXCSUM_TUOFL | E1000_RXCSUM_IPOFL' which was an intentional behavior change but it did not improve this IPv6 use. I will need to do some document reading. Regards, Kevin