Proposal: a revoke() system call

[Bruce Evans]

On Mon, 7 Jul 2008, David Schultz wrote:

> On Mon, Jul 07, 2008, Sergey Babkin wrote:
>>>> Rationale:
>>>>
>>>> In the multithreaded programs often multiple threads work with the
>>>> same file descriptor. A particularly typical situation is a reader
>>>> thread and a writer thread. The reader thread calls read(), gets
>>>> blocked until it gets more data, then processes the data and
>>>> continues the loop. Another example of a "reader thread" would be
>>>> the main thread of a daemon that accepts the incoming connections
>>>> and starts new per-connection threads.
>>>
>>> Have you tried to implement the functionality you're asking for ?
>>>
>>> You'll have to hunt down into all sorts of protocols, drivers
>>> and other code to find the threads sleeping on your fd so you can
>>> wake them.
>>
>> My thinking has been that if close() wakes them up, then things would be
>> inherited from there. The thing I didn't know is that apparently in many cases close()
>> doesn't wake them up.
>
> In Solaris, if you close a file descriptor that has blocked
> readers, the readers wake up and read() returns 0 bytes (EOF).
> (At least this is true if you close the local end of a pipe.)
> It seems like implementing the same behavior in FreeBSD would
> address your problem without introducing a new system call.
> Is there a good reason why this might not be the right thing to do?

Does this happen even for non-last closes of all file types?  Pipes
are too simple :-).

Under FreeBSD, ordinary revoke(2) needs to do wake up all readers and
synchronize with them (preferably without waiting for them), but it
has never done this.  The kernel has no mechanism for finding threads
sleeping or doing i/o on an fd short of what fstat does (searching
half of kmem for hints).  Only a small amount of progress has been
made in fixing this in the 20 years that revoke() has existed.  Most
of the necessary wakeups don't occur.  A few occur accidentally.  So
it is normal for threads to be left active after revoke() completes,
and the progress is mainly that the devfs and conf layers try harder
to prevent deallocation of active data structures for devices in this
state.  The active threads may do some damage when they wake up with
a closed or a new generation of open device, but usually don't.  Tty
drivers use a generation count to prevent some uses of new generations
of opens, but don't check it in enough places.  I haven't noticed any
other class of drivers doing even this much.  Since revoke() is used
mainly on tty devices and the generation count almost works for these,
these bugs are rarely noticed.

Bruce