From nobody Wed Aug 09 13:51:11 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RLWg939ssz4q1Bw for ; Wed, 9 Aug 2023 13:51:17 +0000 (UTC) (envelope-from void@f-m.fm) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RLWg82ymFz4Qsy for ; Wed, 9 Aug 2023 13:51:16 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=jNsf7TyO; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="w CTFkaA"; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 64.147.123.20 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 6C47632002F9 for ; Wed, 9 Aug 2023 09:51:14 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 09 Aug 2023 09:51:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1691589073; x=1691675473; bh=Yrxjf1e6xo/iGLkAGqo1EguNT5894/sk00b sXgBHtEA=; b=jNsf7TyOeRwvgI3rZna2SAuqOPKr2/sj2AyS/eqxwtgdE238/6k CbCGeh3nTbBNn/Vj6FhSIJL38IEuBIz79zW37zgQQ9yWnGfjQFD9eYzakc1BDV0f t1kNsN0PaYC0W3SqiUGq6iX7DIHq2ixPTOSm5g10TQIUxbKQWb45ELBs5uM+gTwh WdnmvUqEVTISb51dVeusSzG4zb/+rXxlRjlJ9LahVXFwe+UlV13jBbBoPosUieVv nI7XNMzIUSBd67ZARBPFiHl7NPOUyZJvwn0/QDPouSMn1yqUbg8ZI9soDc2TVEud vu3iUqQQ9NtVTn9yTmQ7uJZdg5oT1hVEpxg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1691589073; x= 1691675473; bh=Yrxjf1e6xo/iGLkAGqo1EguNT5894/sk00bsXgBHtEA=; b=w CTFkaAeyVUqY2OFx3vgpdD4y1TKwOB1DcABIfrYYJX8dvIP5gn0avg0KDwchm4zf bG1jwDb2UVIr3qvpViLQrbEc9IvTWDN6fmjW/I6AxYykwALb8f1vQIGDM8KCoZIH fvgfIp5acoYvpNcXscgtyBTzKCe028q9hmYCUAFHKe5gBQ4fWEWFgR9hSDwzqDYb lTosuMQHhT580KTbejc0VZXy+yAedPM9c3mc/xRotyy3D1c7TpyuwRiASRu4yJSi /QcUEZW1Hmmkq6AI70ud7Wu2Dfo3Kh6gMzIHnQfCruVeu5TmwQ5draxzazB1ad7j BqeEU1EbvXffwiFuFxySQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrleeggdeikecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtugfgjgesthekre dttddtjeenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgr thhtvghrnhepudetveeggeegveeitedvudduteffffffhfetjefggfegveejffeiudetfe etvdeknecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhep vhhoihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 9 Aug 2023 09:51:13 -0400 (EDT) Date: Wed, 9 Aug 2023 14:51:11 +0100 From: void To: freebsd-hackers@freebsd.org Subject: Re: can sftp be made multi-threaded? Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org References: <3639ADD0-3715-4B2B-9C25-EBF367FC09A0@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3639ADD0-3715-4B2B-9C25-EBF367FC09A0@FreeBSD.org> X-Spamd-Result: default: False [-3.95 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_SHORT(-0.95)[-0.951]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; RWL_MAILSPIKE_EXCELLENT(-0.40)[64.147.123.20:from]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.20]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.20:from]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RLWg82ymFz4Qsy On Wed, Aug 09, 2023 at 02:49:48PM +0200, Juraj Lutter wrote: >Have you played with misc/mbuffer from ports? It might do what you are >looking for (provided that sftp alone isn’t sufficient). I've used mbuffer in a remote zfs snapshot context before but I thought that was to account the bursty nature of sending zfs backups because reading from the disks was bursty. This isn't the case here, as disks are all m.2 nvme connected to pci4 risers. The problem I'm seeing I can see through a tool like speedtest-go and running the test in single-thread mode. If 10 threads are used, I can saturate the connection. If 1 thread is used, the connection speed reported is a small fraction of the actual. I'm hypothesising that throughput-per-thread might be limited either by the ISPs and/or by for example sysctls on the two freebsd systems. I know there's something like third-party patches like HPN(?) that *might* do what I want but for a start I'm not sure they will work with openssh v3. I'm interested how others accomplish secure backup over long distances at near-line-speeds. If they need to recompile base openssh to accomplish it, if they're using v3 and any sysctls or other settings in ssh_/sshd_config --