Mosh regression between 10.x and 11-stable
cgull at glup.org
Sun Aug 14 07:22:55 UTC 2016
On 8/13/16 4:30 AM, Peter Jeremy wrote:
> Hi John,
> Sorry, I got side-tracked.
> On 2016-Aug-12 16:37:15 -0400, John Hood <cgull at glup.org> wrote:
>> >Could I ask you to look at this a little further? On the one hand, it
>> >sure looks like a Mosh issue, and tcdrain() solves it-- but on the other
>> >hand, this is a regression on FreeBSD and we don't see this issue on any
>> >other OS. I'd like to fully understand this and make sure that this is
>> >not a kernel issue.
> It's got me puzzled as well. And it's only getting wierder... The
> following is using an unmodified mosh-1.2.5, built from the port, as
> the server on FreeBSD 11.0-BETA4 r303957. The client is 1.2.4a-1ubuntu1
> on Linux. the standard driver script consistently fails but adding a
> "print" makes it mostly work. Where there's a "[mosh is exiting.]"
> message, it was successful (and would report that there were other
> orphaned servers since I wasn't waiting the 60 seconds for servers to
> die between invocations). For completeness, I've tried ktrace'ing
> mosh-server but can't make it fail when I do so.
I've now managed to reproduce the issue *and* ktrace it (and sshd) on a
VPS, with a single-CPU VM on a badly-oversubscribed VMWare host and OS X
mosh-server shows a straightforward execution trace, the parent
successfully writes the MOSH CONNECT message on stdout, forks and exits.
About a millisecond later, the child starts running, writes the verbose
copyright/etc. message on stderr, and gets EIO (and ignores it).
Shortly thereafter it closes its pty slave fds on stdin/stdout/stderr.
It continues normally from there.
In that millisecond, the sshd trace shows that it catches SIGCHLD,
writes utmp info, wait()s for a child and gets mosh-server's pid, and
does a final read() that returns 0 bytes. It then closes the pty
master, without doing revoke() or any ioctls.
So the pty driver is clearly dropping the MOSH CONNECT message. It's a
lot less clear whether that's a bug. On the one hand,
states that output should be drained on final close (which in this case
is done by the forked mosh-server). On the other hand, the login shell
is session leader, and
requires that the terminal be disassociated from the session when it exits.
I suspect that this problem is most visible on single-core machines,
because on a multi-core machine the pty driver, mosh-servers, and sshd
will run with different ordering, and I suspect that the mosh-server
child gets to close the pty slave before sshd closes the pty master.
I wrote a Xenix serial driver ages ago and I can see arguments for
either draining or dropping final output. And Mosh's behavior is a
little questionable here. So I'm not sure whether to call this a kernel
pty bug or not. But the workaround in Mosh is easy enough, tcdrain(),
so I'm doing that anyway.
More information about the freebsd-current