Background processes setting O_NONBLOCK on ttys

[Stephen McKay]

[You may see this message twice as it got stuck in the ^$&#@ moderation queue.]

Hi!  I'm running FreeBSD 4.11-RC2 on an Athlon 2100+ with 768MB of ram and
software mirrored 160GB Seagate disks.  All pretty straightforward so far.

I'm currently a couple of DAYS into compiling Open Office 1.1.3, and while
I can tell you all sorts of stories of trouble and woe, I want to concentrate
on just one small aspect.

I commonly background all builds, this port included, and then I continue
doing whatever I like in that window.  In this case, things start dying.

For example, "tail -f LOG" works for a while, and then mysteriously exits.
It's something I've not seen before in 20 years of Unix!

After some tracing, I have worked out that the tty is being alternately
set to nonblocking and back to normal hundreds of times during the compilation
of Open Office, and that's what is killing tail.

Tail does not expect stdout to be nonblocking, and will exit if it ever
receives EAGAIN.  This is because putchar() in stdio passes this back up as
an error.  This causes tail to exit (return code 1).

I haven't found which program is doing the nonblocking flipping yet.  I expect
I will eventually.  What I initially find is quite odd is that a background
process is permitted to modify the tty like that.

I checked the flow of control of fcntl() as it sets O_NONBLOCK on a file
descriptor and eventually control passes to ttioctl() which includes code
to prevent background processes messing with the tty.  (At least, I think
it does.  This is code involving many indirections.)  But, for whatever
reason, setting non-blocking mode (which is called FIONBIO at this point)
is not one of the set of things considered as "modification" of the tty,
so it is passed through unchallenged.

I intend to add the setting of non-blocking mode as one of the things that
will cause a background process to receive SIGTTOU and see if my life
improves.  If it works as well as I hope, I'll push for it being official.

I also think this mechanism may be behind occasional failures of vi that
I (and some others here and there) have seen over the last few years.

By the way, the code seems to be the same in -current and 5.x so this will
apply there if it turns out to be sensible.

Also, stdio in -current is unchanged, and will report EAGAIN as a failure
for putchar().  This is the documented behaviour (at least it is in the
Single Unix Specification).  Should tail be coded such that EAGAIN on
stdout is detected and worked around?  That seems like too much to expect
tail (and every other ordinary utility) to know about, so I hope my other
efforts yield fruit.

Stephen.