ssh over WAN: TCP window too small

[Kevin Oberman]

On Tue, Aug 25, 2015 at 2:12 PM, Chris Stankevitz <chris at stankevitz.com>
wrote:

> On 8/25/15 4:11 PM, Bjoern A. Zeeb wrote:
>
>>
>> On 25 Aug 2015, at 22:47 , Chris Stankevitz <chris at stankevitz.com> wrote:
>>>
>>> Can anyone recommend some tools/tricks to figure out what in FreeBSD
>>> and/or
>>>
>> >> base SSH is limiting the send/recv buffer and/or TCP window?
>
>>
>> if you have the memory, try these sysctls:
>>
>> kern.ipc.maxsockbuf=146800640
>> net.inet.tcp.recvbuf_max=67108864
>> net.inet.tcp.sendbuf_max=67108864
>>
>
> Bjoern,
>
> Thank you for the reply.  Before your suggestion my sysctls are:
>
>  kern.ipc.maxsockbuf=2097152
>  net.inet.tcp.recvbuf_max=2097152
>  net.inet.tcp.sendbuf_max=2097152
>
> Each of these is much larger than the limit I am experiencing (~64,000).
> So I [naively] expect changing these values will have no effect.  Let me
> try...
>
> ... okay... sure enough the sysctrl changes you suggest did not change the
> 64,000 bytes-in-flight limit I am experiencing.
>
> Thanks for the idea (and keep 'em coming!),
>
> Chris
>

My former employer, ESnet, was heavily involved in moving very large
amounts of data (petabytes) over very long (intercontinental), very fat
(100G) pipes. In an effort to improve customer satisfaction they have done
extensive research into the issues involved and have published much of it
at http://fasterdata.es.net. In particular, they have documented the issues
with ssh over long latency links at
http://fasterdata.es.net/data-transfer-tools/say-no-to-scp/.

It is VERY hard to get good performance on high latency links in the bast
of cases and, unfortunately, ssh/scp makes it not the best of cases.
--
Kevin Oberman, Goat herder and Retired Network Engineer