From nobody Sun Feb 25 02:52:11 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tj7Zd3Hfcz540kL for ; Sun, 25 Feb 2024 02:52:25 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tj7Zc4VVGz56lg for ; Sun, 25 Feb 2024 02:52:24 +0000 (UTC) (envelope-from nonesuch@longcount.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=longcount.org header.s=google header.b=ff8rhV66; dmarc=none; spf=pass (mx1.freebsd.org: domain of nonesuch@longcount.org designates 2607:f8b0:4864:20::731 as permitted sender) smtp.mailfrom=nonesuch@longcount.org Received: by mail-qk1-x731.google.com with SMTP id af79cd13be357-787c97deabdso25014385a.3 for ; Sat, 24 Feb 2024 18:52:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount.org; s=google; t=1708829543; x=1709434343; darn=freebsd.org; h=to:in-reply-to:references:message-id:date:subject:mime-version:from :content-transfer-encoding:from:to:cc:subject:date:message-id :reply-to; bh=qJ89aM7WIoVUpZwAfhIu6JJYErMPWXe/8FT7FdSYqTo=; b=ff8rhV660b5FbKHAs/DINzzr6Ckg0xLoxOgsL/X1NnBDtZruUbtvGkXozlMQMXI2sk WoAkZH750Umjymd7l026ok+mn4alg5JjOsQLPnI7yAaSNyatA61ihSiTCaXrgwjK3oTi OIfKI9MiYvu+ACY7j2nMTQThvNTU982bPcSTZ2aoLTyT4agyZnkxdHSbBiUmlIVz90rz o+i1dZbwkk3k6sfyQRhQkSUgpRgX5Ovxp/lR8ZXf0Q/uaqP1EnRbMhuIC03Lrz3rH0w6 YGlWrNu+rStng96lrcTB+jjZeQK2/oOR5xKaDfgsBoAjgbR24jGTuAQkUhp3VoAPqaAJ IbZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708829543; x=1709434343; h=to:in-reply-to:references:message-id:date:subject:mime-version:from :content-transfer-encoding:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=qJ89aM7WIoVUpZwAfhIu6JJYErMPWXe/8FT7FdSYqTo=; b=JZNXnIS2ODauybScRSgcYfgiJyRCBej3FlCmCfphaCmSTRHNkQr9sXhDxW7YLZBXXi 9hHV6FDhp6mKNSeRPxJiNazShWcyJN702VXtz79lwwgqHaxp4DXUhzDKvEp9uLqOGqQw gPE9bYx+99MSXUIatImPS5eEhZie8zz9+86SEpuD85oqSadPhLogakOuhImpYdCzafpR wsPi0tQJe6OBJWjUtNNGDFwe19izftIGSazOqTAJOBOSHa2aHcWMYv8qi1Rpg+JRKOhX Erunp+KSyLiy6ge3yyBWAlQqyvm6fpCIsvXruNmO0Z2eLCqJE/8hkTHnqK1ymfozAAXx ljqw== X-Gm-Message-State: AOJu0YyamC3BcHuXnP/YI3VYO0w1JGpNDvSLIAmQMPalLrtva+4acV0p Xleq8Qmv9A2h1IVM79g452dCmVXClSSgiPJO9lj8bDkrtSeWyJ4MQ8eNJfKBpO4LhRszA6VaSFn AUaA= X-Google-Smtp-Source: AGHT+IFtKgqZbhfrRzQil50vY9IILmhgy2QmrH+Bm5G80LfR6VEQUVe29/J5/cnemA1kejeR4nwQRw== X-Received: by 2002:a05:620a:4d11:b0:787:8ed8:f2aa with SMTP id wa17-20020a05620a4d1100b007878ed8f2aamr3685620qkn.35.1708829542840; Sat, 24 Feb 2024 18:52:22 -0800 (PST) Received: from smtpclient.apple (pool-70-104-128-251.nycmny.fios.verizon.net. [70.104.128.251]) by smtp.gmail.com with ESMTPSA id bm22-20020a05620a199600b0078742e741cfsm1134788qkb.61.2024.02.24.18.52.22 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 24 Feb 2024 18:52:22 -0800 (PST) Content-Type: multipart/alternative; boundary=Apple-Mail-E0DD2C68-F6FB-45A9-BD26-CE8E9A032E65 Content-Transfer-Encoding: 7bit From: Mark Saad List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: NFS performance with 10GBase-T Date: Sat, 24 Feb 2024 21:52:11 -0500 Message-Id: <9F370420-52DC-4895-B2B3-5254B6E470D0@longcount.org> References: <034cc6ea-26d8-4520-879a-672459832407@fsfe.org> In-Reply-To: <034cc6ea-26d8-4520-879a-672459832407@fsfe.org> To: freebsd-net@freebsd.org X-Mailer: iPhone Mail (20H240) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[longcount.org:s=google]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DKIM_TRACE(0.00)[longcount.org:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[longcount.org]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; APPLE_IOS_MAILER_COMMON(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::731:from] X-Rspamd-Queue-Id: 4Tj7Zc4VVGz56lg --Apple-Mail-E0DD2C68-F6FB-45A9-BD26-CE8E9A032E65 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > On Feb 24, 2024, at 7:18 PM, Hannes Hauswedell wro= te: >=20 > =EF=BB=BFHi everyone, >=20 > I am coming here from > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D2771971160 >=20 > TL;DR: >=20 > * I have a FreeBSD14 Server and Client with an Intel X540 (ix) adaptor eac= h. > * I am trying to improve the NFS throughput. > * I get 1160 MiB/s via nc, but only ~200 MiB/s via NFS. > * Increasing rsize and wsize to 1 MiB increases throughput to 395 MiB/s > * But a Linux client achieves 560-600 MiB/s with any rsize. > * The mtu is set to 9000 but this doesn't make a difference for the FreeBS= D client (it does make a difference for Linux). >=20 > I assume < 400 MiB/s is not the expected performance? Do you have any advi= ce on debugging this? >=20 > Thank you for your help, > Hannes >=20 Hi Hannes, The link didn=E2=80=99t work . Can you tell us more about the file system ,= what type of disks . Details about the server , hardware details. If you w= ant post a copy of your dmesg to https://dmesgd.nycbug.org/index.cgi that ca= n help too . Also what kinds of clients are you using ? Do you have any relevant network stats or a topology/ diagram of the shape o= f things ?=20 Mark saad | nonesuch@longcount.org= --Apple-Mail-E0DD2C68-F6FB-45A9-BD26-CE8E9A032E65 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable


On Feb 24, 2024, at 7:18 PM, Hannes Hauswedel= l <h2+lists2024@fsfe.org> wrote:

=EF=BB=BFHi everyone,
=
I am coming here from
https://bugs.freebsd.= org/bugzilla/show_bug.cgi?id=3D2771971160

T= L;DR:

* I have a FreeBSD14 Server and Clien= t with an Intel X540 (ix) adaptor each.
* I am trying to imp= rove the NFS throughput.
* I get 1160 MiB/s via nc, but only= ~200 MiB/s via NFS.
* Increasing rsize and wsize to 1 MiB i= ncreases throughput to 395 MiB/s
* But a Linux client achiev= es 560-600 MiB/s with any rsize.
* The mtu is set to 9000 bu= t this doesn't make a difference for the FreeBSD client (it does make a diff= erence for Linux).

I assume < 400 MiB/s i= s not the expected performance? Do you have any advice on debugging this?

Thank you for your help,
Hann= es


Hi Hannes,
  The link didn=E2=80=99t work . Can you tell us more about the file= system , what type of disks . Details about the server , hardware details. &= nbsp;If you want post a copy of your dmesg to https://dmesgd.nycbug.org/index.cgi that can= help too .

Also what kinds of clients are you usin= g ?

Do you have any relevant network stats or a top= ology/ diagram of the shape of things ? 

Mark saad | nonesuch@longcount.org
=
= --Apple-Mail-E0DD2C68-F6FB-45A9-BD26-CE8E9A032E65--