From nobody Mon Jan 30 21:59:30 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4P5MXx46Vjz3cjFD for ; Mon, 30 Jan 2023 21:59:45 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (plan-b.pwste.edu.pl [IPv6:2001:678:618::40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "plan-b.pwste.edu.pl", Issuer "GEANT OV RSA CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4P5MXw1Bqgz46Ry for ; Mon, 30 Jan 2023 21:59:43 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=plan-b.pwste.edu.pl header.s=plan-b-mailer header.b=cpFNLlh2; spf=none (mx1.freebsd.org: domain of zarychtam@plan-b.pwste.edu.pl has no SPF policy when checking 2001:678:618::40) smtp.mailfrom=zarychtam@plan-b.pwste.edu.pl; dmarc=pass (policy=none) header.from=plan-b.pwste.edu.pl Received: from [IPV6:2a02:22e0:cf00:1ff:c526:f4c3:3a62:2b92] (mzar@[IPv6:2a02:22e0:cf00:1ff:c526:f4c3:3a62:2b92]) (authenticated bits=0) by plan-b.pwste.edu.pl (8.17.1/8.17.1) with ESMTPSA id 30ULxW47045341 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Mon, 30 Jan 2023 22:59:32 +0100 (CET) (envelope-from zarychtam@plan-b.pwste.edu.pl) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=plan-b.pwste.edu.pl; s=plan-b-mailer; t=1675115973; bh=5ZS9u3O8sJ+kQ44Wgm7uVgxtQ+Y7bvCkokJkQpcdY7g=; h=Date:To:References:From:Subject:In-Reply-To; b=cpFNLlh27Rcw/jxp6XPvtCZTsceKfTk93f/UDV36STP95daZdVgFodJaWpH7FCFcs mZ9D5j9qiGwbIVg90pqzTuOy98sUYbiEi4GNR0UBHWXvWUcGCs9pXy72yT2PbVdxD1 +4z85OKV/aMBJ1wWlojWMjFSuExi6j6+1cz95XzU0elrIxfNPmFv53LLEUFGXXNle0 u1rNt8ofJw4aLgmv49ZFrUT1ItE0Ad8IANypvMyYQUn9NQZFSr5Yz94xeFxQB4+sXj 4nK9hUzxkBZXepS2d8e5JPOW3eMTuVTTh86nIid5Ge0EeXZD4wp0M079e7mq2s+5zG zNPugyfkL0zgg== Message-ID: <4ed8b724-041f-f561-ae60-ab966aefbb68@plan-b.pwste.edu.pl> Date: Mon, 30 Jan 2023 22:59:30 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 To: stable@freebsd.org References: <95EDCFCA-7E3F-458F-85A6-856D606B9D98@gromit.dlib.vt.edu> Content-Language: en-US From: Marek Zarychta Subject: Re: Slow WAN traffic to FreeBSD hosts but not to Linux hosts---how to debug/fix? In-Reply-To: <95EDCFCA-7E3F-458F-85A6-856D606B9D98@gromit.dlib.vt.edu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-2.80 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[plan-b.pwste.edu.pl,none]; R_DKIM_ALLOW(-0.20)[plan-b.pwste.edu.pl:s=plan-b-mailer]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[stable@freebsd.org]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:206006, ipnet:2001:678:618::/48, country:PL]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[plan-b.pwste.edu.pl:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4P5MXw1Bqgz46Ry X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N W dniu 30.01.2023 o 22:17, Paul Mather pisze: > TL;DR: When working from home, I can max out my residential 200 Mbit network connection when downloading from remote Linux hosts at $JOB but only manage about 20% of my max residential connection speed when downloading from remote FreeBSD hosts at $JOB. When at $JOB, both FreeBSD and Linux hosts have no problem saturating their GbE connections transferring between each other. Why is this and how can I debug and fix it? > > I have a 200 Mbit residential cable connection (Xfinity, 200 Mbit down/~10 Mbit up). I've noticed recently that I can easily get 10--20 MB/s download speeds when transferring data from Linux hosts at work but when I try to download that same data from the FreeBSD hosts I use the speed usually tops out at 3--4 MB/s. These are Linux and FreeBSD hosts that are on the same subnet at work. Transfers from the FreeBSD hosts at work (within-subnet and within-site) are fine and match those of the Linux hosts---often 112 MB/s. So, it just appears to be the traffic over the WAN to my home that is affected. The WAN path from home to this subnet is typically 15 hops with a typical average ping latency of about 23 ms. > > The FreeBSD hosts are a mixture of -CURRENT, 13-STABLE, and 13.1-RELEASE. I had done some TCP tuning based upon the calomel.org tuning document (https://calomel.org/freebsd_network_tuning.html), but removed those tuning settings when I noticed the problem but the problem still persists. The only remaining customisation is that the 13-STABLE has "net.inet.tcp.cc.algorithm=cubic". (I notice that -CURRENT now has this as default so wanted to try that on 13-STABLE, too.) The FreeBSD systems are using either igb or em NICs. The Linux systems are using similar hardware. None has a problem maintaining local GbE transfer speeds---it's only the slower/longer WAN connections that have problems for the FreeBSD hosts. > > It seems that Linux hosts cope with the WAN path to my home better than the FreeBSD systems. Has anyone else noticed this? Does anyone have any idea as to what is obviously going wrong here and how I might debug/fix the FreeBSD hosts to yield faster speeds? My workaround at the moment is to favour using the remote Linux hosts for bulk data transfers. (I don't like this workaround.) > > Any help/insight is gratefully appreciated. > > Cheers, > > Paul. > While playing with different mod_cc(4) might bring some improvement, to get a real boost I'd suggest enabling tcp_rack(4) if feasible. -- Marek Zarychta