Re: -current dropping ssh connections

On Mon, Jun 26, 2023 at 07:38:22PM -0700, Mark Millard wrote:
> On Jun 26, 2023, at 19:00, bob prohaska <fbsd@www.zefox.net> wrote:
> 
> > On Thu, Jun 22, 2023 at 11:38:50PM +0100, Jamie Landeg-Jones wrote:
> >> bob prohaska <fbsd@www.zefox.net> wrote:
> >> 
> >>> That seems worth a try.
> >>> The notion of an ssh escape (~. in this case) finding its way into the data stream is new to me.
> >> 
> >> Thinking again, that looks like corruption coming down the ssh connection.
> >> For the ssh escape char to affect anything (note it needs to be preceeded
> >> by a new line) it would have to be sent up the line.
> >> 
> >> For an example, assuming ssh has the default escape char, look at the difference between:
> >> 
> >> printf 'pwd;\n~.;echo sleeping.; sleep 5' | ssh -tt user@host
> >> 
> >> and with the same printf, but no escape char:
> >> 
> >> printf 'pwd;\n~.;echo sleeping.; sleep 5' | ssh -tt -e none user@host
> >> 
> >> You can see how the former closes the connection due to the \n~.
> >> 
> >> (The -tt forces a terminal/interactive session to be set up - normally, as we are
> >> piping input to ssh in this case, the terminal isnt set up, and the escape character
> >> isn't used - it's only recongnised in interactive sesions by default)
> >> 
> >> I personally have "EscapeChar none" in my ssh_config, but I suspect this
> >> is probably not the issue here, still, can't hurt to try it!
> > 
> > Indeed, connection was dropped with escape character set to none.
> > There does seem to be a some dependence on system load. Connections
> > survive at light or no load and drop when the sshd side is busy. 
> 
> So how busy is the disk media (really the USB bus) from
> somewhat before the failures happen to the failure point?
> The question applies separately to both systems involved
> in the ssh session.
>

I've no quantitative measure. Is there any command I could 
add to the swap activity logger that can report USB load?


> I'll remind that on a RPi2B v1.1, RPi2B v1.2, or RPi3B*
> the Ethernet and the 4 USB ports all share the one USB2
> path at an internal stage for the USB subsystem, if I
> remember right.
> 
> Saturating that USB2 path with data transfer bytes
> to/from disk media might be able to block other activity
> long enough to cause the Ethernet communications to have
> problems for all I know. (Amount of data queued for
> transfer would be relevant.)
> 
> This might be a reason to avoid settings for the
> combination vm.pfault_oom_attempts and vm.pfault_oom_wait
> to values that could contribute to a sustained period of
> saturating the USB channel.
>

At this point the vm.* settings are default, with no
warnings on the serial console apparent.. 

> (I have no detailed knowledge of how the tradeoffs work
> for the competing uses of the USB channel. So I can not
> validly claim to know that the above is realistic.)
>

It is tempting to think USB saturation would account 
for the trouble. A tool similar to gstat, but for
USB as a whole, would be helpful if it exists.
 
It's perhaps notable that the dropped connections are
seen in connection with usb-serial adapters more than
interactive connections running shells.

Thanks for reading!

bob prohaska