From nobody Sun Feb 25 23:06:23 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TjfWV5Mmvz5BXF6 for ; Sun, 25 Feb 2024 23:06:30 +0000 (UTC) (envelope-from h2+lists2024@fsfe.org) Received: from mail3.fsfe.org (mail3.fsfe.org [IPv6:2a02:16d0:1004:5a00:f5f3::90]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TjfWT5G1bz4hhw for ; Sun, 25 Feb 2024 23:06:29 +0000 (UTC) (envelope-from h2+lists2024@fsfe.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fsfe.org header.s=2023072501 header.b=bqr3LTv6; dmarc=pass (policy=none) header.from=fsfe.org; spf=pass (mx1.freebsd.org: domain of h2+lists2024@fsfe.org designates 2a02:16d0:1004:5a00:f5f3::90 as permitted sender) smtp.mailfrom=h2+lists2024@fsfe.org Message-ID: <714c3e34-575f-41ab-8149-6e3019f91bf1@fsfe.org> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2023072501; t=1708902398; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5HGUkSai6nNr1es6c+SQpaJ4ROOWPutxsbueHMZp4aM=; b=bqr3LTv6YOMOIHje3bwtOaq6TwRt0K/IlG/KHItPsNqEM6XjhFuxQZyjANQcTS2Hgv1FFX ZtsYHMHHeLQduf/Lr4vfectaVef6qB38bHcs00ActD5mrtzYRl0RUQ+ZFUnkf5s5+CUvV3 2Jk6wR/GzphRT20WyqvXAXa9b161QIg= Date: Mon, 26 Feb 2024 00:06:23 +0100 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: NFS performance with 10GBase-T Content-Language: en-US To: freebsd-net@freebsd.org References: <034cc6ea-26d8-4520-879a-672459832407@fsfe.org> <9F370420-52DC-4895-B2B3-5254B6E470D0@longcount.org> From: Hannes Hauswedell Organization: Free Software Foundation Europe In-Reply-To: <9F370420-52DC-4895-B2B3-5254B6E470D0@longcount.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.99 / 15.00]; DWL_DNSWL_MED(-2.00)[fsfe.org:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[fsfe.org,none]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[fsfe.org:s=2023072501]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:49855, ipnet:2a02:16d0::/32, country:DE]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; HAS_ORG_HEADER(0.00)[]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TAGGED_FROM(0.00)[lists2024]; RCVD_IN_DNSWL_NONE(0.00)[2a02:16d0:1004:5a00:f5f3::90:from]; DKIM_TRACE(0.00)[fsfe.org:+] X-Rspamd-Queue-Id: 4TjfWT5G1bz4hhw >> Hi everyone, >> >> I am coming here from >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=2771971160 >> >> TL;DR: >> >> * I have a FreeBSD14 Server and Client with an Intel X540 (ix) adaptor >> each. >> * I am trying to improve the NFS throughput. >> * I get 1160 MiB/s via nc, but only ~200 MiB/s via NFS. >> * Increasing rsize and wsize to 1 MiB increases throughput to 395 MiB/s >> * But a Linux client achieves 560-600 MiB/s with any rsize. >> * The mtu is set to 9000 but this doesn't make a difference for the >> FreeBSD client (it does make a difference for Linux). >> >> I assume < 400 MiB/s is not the expected performance? Do you have any >> advice on debugging this? >> >> Thank you for your help, >> Hannes >> > > Hi Hannes, thank you for the reply! >   The link didn’t work . Sorry, no clue how that copy-pasta happened. The link Michael posted is correct: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277197 > Can you tell us more about the file system , > what type of disks . 4x4TiB NVME (Lexar NM790) raidz1 The server can maintain speeds over 900 MiB/s copying hundreds of Gigabytes from one dataset to another including compression and encryption on both ends, so I don't think the server setup is a problem. nc manages to send data at 1100 MiB/s from the server to the client. > Details about the server , hardware details.  If > you want post a copy of your dmesg to > https://dmesgd.nycbug.org/index.cgi >  that can help too . https://dmesgd.nycbug.org/index.cgi?do=view&id=7527 If there is anything else that would help, please let me know! > Also what kinds of clients are you using ? The FreeBSD14 client is an amd64 machine with the same network adaptor as the server (Intel X540 T1). I have set up a dual-boot with Devuan GNU/Linux. As reported above, Linux achieves 600 MiB/s out of the box. > Do you have any relevant network stats or a topology/ diagram of the > shape of things ? The client and server are connected via CAT7 cables and a NETGEAR switch. I could look up the model later, but it can probably be ruled out if other tools/OS achieve higher throughput? Best regards, Hannes -- Dr. Hannes Hauswedell C++ Software Engineer https://hannes.hauswedell.net