Re: speeding up zfs send | recv (update)

I use mbuffer for this purpose. I'm only on protected "inside" links
even if cross-router (ie not just a switching fabric) which may make
my use-case a poor fit for your need.

mbuffer has several strategies around how much buffer to use, what
rate limits to apply, I believe can do tweaks to use scatter-gather
models (the necessary ioctls to turn things on &c but these are almost
certainly tuned to linux)

you also need to look at your ethernet card offload. Normally
beneficial. it can interact badly with in-kernel models of end-to-end
flow, because it's performing its own work "on your behalf"

I sort of miss having a "null" cipher in SSH. I didn't entirely
understand why it got stripped out: the 'who are you" initialisation
authorisation is beneficial, even if the datastream is (deliberately)
unprotected. the RC2 fallback didn't seem to impose much cost, but
thats gone too now.

-G

On Thu, Feb 23, 2023 at 7:43 AM Freddie Cash <fjwcash@gmail.com> wrote:
>
> [Sorry for top part, GMail sucks for replies.]
>
> If this is a LAN or private WAN where you trust the network, piping the send stream through netcat will remove ssh from the equation.
>
> That's what we switched to using once it became almost impossible to get the "none" cipher working with ssh on FreeBSD.
>
> We use ssh to connect to the remote server and enable a netcat listener on port X, then pipe the send through netcat to the remote system on port X. That way it's logged and uses ssh for authentication.
>
> We easily saturate gigabit links between our ZFS systems using netcat.
>
>
>
> Cheers,
> Freddie
>
> Typos due to smartphone keyboard.
>
> On Wed., Feb. 22, 2023, 1:31 p.m. Miroslav Lachman, <000.fbsd@quip.cz> wrote:
>>
>> On 22/02/2023 22:08, mike tancsa wrote:
>> > On 2/22/2023 4:03 PM, Miroslav Lachman wrote:
>> >> Interresting numbers. I think I am the only one who get best speed
>> >> with chacha20-poly1305@openssh.com
>> >>
>> >>
>> >> It seems the speed of SSH is limited by single core performance which
>> >> is very poor on this machine (Intel(R) Pentium(R) Dual  CPU E2160).
>> >> Even if CPU has 50% idle, ssh runs on 99.8% of single core.
>> >
>> > The CPU I have has
>> > aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS> on motherboard
>> >
>> > which probably helps.
>>
>> That explains it
>> aesni0: No AES or SHA support.
>>
>> >> I know there were some HPN patches to ssh, beside that is there any
>> >> option I can try to use less CPU?
>> >>
>> >> I will play with cpuset to pin ssh on one core and everything else on
>> >> the other core.
>> >
>> > It looks like you are running into a CPU bottleneck TBH
>>
>> Yes. Pinning on cores with cpuset helps a bit (about +3MiB/s) but
>> without some tweaks on ssh I will not gain more speed :(
>>
>> Thank you for your help!
>>
>> Miroslav Lachman
>>
>>