From nobody Wed Feb 01 02:46:10 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4P65s35SRDz3bp1H for ; Wed, 1 Feb 2023 02:46:15 +0000 (UTC) (envelope-from 2yt@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4P65s25LM9z4Js5 for ; Wed, 1 Feb 2023 02:46:14 +0000 (UTC) (envelope-from 2yt@gmx.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of 2yt@gmx.com designates 212.227.15.18 as permitted sender) smtp.mailfrom=2yt@gmx.com; dmarc=pass (policy=none) header.from=gmx.com Received: from [192.168.160.51] ([47.5.85.140]) by mail.gmx.net (mrgmx005 [212.227.17.184]) with ESMTPSA (Nemesis) id 1N7iCW-1ob0mE3Rhq-014iEY for ; Wed, 01 Feb 2023 03:46:13 +0100 Message-ID: Date: Tue, 31 Jan 2023 19:46:10 -0700 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: Slow WAN traffic to FreeBSD hosts but not to Linux hosts---how to debug/fix? Content-Language: en-US References: <95EDCFCA-7E3F-458F-85A6-856D606B9D98@gromit.dlib.vt.edu> <4ed8b724-041f-f561-ae60-ab966aefbb68@plan-b.pwste.edu.pl> <282AF730-E5E0-4A50-9F47-E7301B36E5C8@gromit.dlib.vt.edu> <2ed582b9-b544-74bb-2047-99d04924b46b@plan-b.pwste.edu.pl> To: freebsd-stable@freebsd.org From: David <2yt@gmx.com> In-Reply-To: <2ed582b9-b544-74bb-2047-99d04924b46b@plan-b.pwste.edu.pl> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K1:I+4C0vuoqifr2V19oLjaiB0XRApbc/R6rpRIQIFRqyxsNplRrWD K9y4YvujnfwMJi3gr1yXroZAK7VhbUYnsvGL9yC9tW0P1IJaTWRal6O0O4Ql7Nel2RnLY0a 5BZqmTILj3Kjya2wZzmFfc16xrdx2z6ufSf/kOjHA12AaOK05YaoZZ0ScuoKjkxXBNKLA9G /UTWDTfxeSnd0vuzYS0sA== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:GH1TzAdr2ko=;YdEzjNV6dfVvg8jSSKTvsFXKelH 4jqKIfyqGMLnrQaFKFGrE3FZuoelSmrd+Iiqx02WR7/p8I+M4PSwhcq2j/NICVgfNskyfo12V ziDRRYN2Sj9X8fuujtIvogM+t7xHzeOgYloF3Bds9ylDUBkKZNFyr4hF2L/M7H7uB1veGIxR4 nyQgXvNvqqw3Ah+kCoVx5B2ke2zeGl7xktzTVT75u/pUCGtiqmH40XpEKP23MGMGobfbAAmn2 HdAZL5kTuR8FiSuw9f+gon903YdmXX9NmQkprFlcOzDqCrPG/9o33KfHRIhB0ysGps5ni/KF8 wU+Nf9kgkDJ4NlgivurEWngsG0v+obXahH5YGU0VknKF09YqhgDyanpVTqOtCiUWJBBdTPBhk U15shJzkavbyWHH04i7957E80i7ea117eYDdjMAz2OuX4qRmhx5I4JPEmhRpkMK7DSvkBkVzO tcMahS7Ej+97/f0Jl4u9/bsNHCv5M8bwJOEdFlYlPTnVb8wVSoJfS/660vZwjidPmc4wjmgcs ya3BjThxSOVQvdqwa5sB3BEAkGoFDP5xwyAtb05CEPP/7+wpkeKAXpPYAuDl8E25COtw8Ibxj q1HmuAvaUDbCALkdKZOWJK9IR+PdNcuh4Kz5PYesLHS9R8PT72oYNK6fHhmYmpbJd3nRuM8su NVgsu8pbBBWO1weCDNvzUDqxsIAPWDBIut6rpNyp4+/ML4lRfU47kRCUyMKhtX7BTA7cuRjeZ ENmoxAmvUCOxiM/J0hH+SnLgMCKlB2RtUHSFjTDtzrneeDF4UiWxhxavv3bkmOO1lu3tdtaZQ DmGK6Bfe1M6OZwqdFy4iYv+6FnovP5VmbiDVRVqC/SWRd+0YPCqHOqt+pwGU1JmdBCa4Qq5Wr TLBwsHd2fy3ayubqnhesglCDaDRBPQSjcYmB9x+okTK1p4BphGBLO8LRB+ECgqiMBAXojvs1B v6nTrO7Yv68Iobaz/VWOVPMWEWc= X-Spamd-Result: default: False [-0.73 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_SPAM_LONG(0.93)[0.932]; NEURAL_HAM_SHORT(-0.76)[-0.761]; DMARC_POLICY_ALLOW(-0.50)[gmx.com,none]; R_SPF_ALLOW(-0.20)[+ip4:212.227.15.0/24:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.15.18:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[212.227.15.18:from]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmx.com]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmx.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4P65s25LM9z4Js5 X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N On 1/31/23 13:38, Marek Zarychta wrote: > W dniu 31.01.2023 o 19:31, Paul Mather pisze: >>> While playing with different mod_cc(4) might bring some improvement, >>> to get a real boost I'd suggest enabling tcp_rack(4) if feasible. >> >> I am interested in trying this out, but believe it is more feasible in >> my case for the -STABLE and -CURRENT systems I am using, not so much >> for the -RELEASE systems that are kept up to date via binary >> freebsd-update updates.  My reading of the tcp_rack(4) man page is >> that you have to build a custom kernel as, unlike the cc_* congestion >> control algorithms, the loadable tcp_rack module is not built by >> default.  Is that an accurate reading? >> > Yes, this gift from Netflix is probably better suited for -STABLE and > -CURRENT as easier to set up there. There is an excellent, up-to-date > article about it by Klara Systems writers[1]. From my experience > tcp_rack(4) is well suited for congested, lossy or redundant network > paths where loses, duplicated packets or races between packets occur. > Not a panacea, but very performant TCP stack based on the _fair_ > algorithm. In some instances, it might help you to saturate the > bandwidth of the link. TCP algo can be loaded/unloaded/changed on the > fly. In FreeBSD 14-CURRENT you can change it on an active socket with > tcpsso(8) utility, in FreeBSD 12 and 13 you have to restart the app > bound to the socket. > Please feel free to play with TCP stacks and congestion algos with the > help of benchmarks/iperf3 to find out what prevents the link from being > saturated and give us some feedback here. > > [1] https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/ > > Cheers > I compiled a custom kernel (releng/13.1) and followed Klara Systems instructions. The results are quite good. I would hope the RACK stack will be included in the upcoming 13.2 release as it is a significant upgrade. Testing: ------------------------------------ All computers are running FreeBSD 13.1. File transfer done with SCP. Route to HOST has 17 hops and 110ms pings. Connection limited to 100mb. Congestion tonight was highly variable, so the data below is very rough. All CC algorithms did better under RACK TCP Stack freebsd (default) ====================================== htcp 100% 315MB 3.7MB/s 01:24 htcp 100% 315MB 3.1MB/s 01:41 htcp 100% 315MB 2.6MB/s 02:03 newreno 100% 315MB 4.8MB/s 01:05 newreno 100% 315MB 1.4MB/s 03:48 newreno 100% 315MB 1.7MB/s 03:01 cubic 100% 315MB 8.3MB/s 00:37 cubic 100% 315MB 1.1MB/s 04:49 cubic 100% 315MB 1.9MB/s 02:44 TCP Stack RACK ====================================== htcp 100% 315MB 3.9MB/s 01:19 htcp 100% 315MB 8.6MB/s 00:36 htcp 100% 315MB 10.6MB/s 00:29 newreno 100% 315MB 10.5MB/s 00:30 newreno 100% 315MB 9.1MB/s 00:34 newreno 100% 315MB 7.6MB/s 00:41 cubic 100% 315MB 4.1MB/s 01:16 cubic 100% 315MB 8.6MB/s 00:36 cubic 100% 315MB 7.3MB/s 00:42 -- David