close(2) while accept(2) is blocked

[John Baldwin]

On Thursday, March 28, 2013 12:54:31 pm Andriy Gapon wrote:
> 
> So, this started as a simple question, but the answer was quite unexpected to me.
> 
> Let's say we have an opened and listen-ed socket and let's assume that we know
> that one thread is blocked in accept(2) and another thread is calling close(2).
> What is going to happen?
> 
> Turns out that practically nothing.  For kernel the close call would be almost a nop.
> My understanding is this:
> - when socket is created, its reference count is 1
> - when accept(2) is called, fget in kernel increments the reference count (kept in
> an associated struct file)
> - when close(2) is called, the reference count is decremented
> 
> The reference count is still greater than zero, so fdrop does not call fo_close.
> That means that in the case of a socket soclose is not called.
> 
> I am sure that the reference counting in this case is absolutely correct with
> respect to managing kernel side structures.  But I am not that it is correct with
> respect to hiding the explicit close(2) call from other threads that may be
> waiting on the socket.
> In other words, I am not sure if fo_close is supposed to signify that there are no
> uses of a file, or that userland close-d the file.  Or perhaps these should be two
> different methods.
> 
> Additional note is that shutdown(2) doesn't wake up the thread in accept(2)
> either.  At least that's true for unix domain sockets.
> Not sure if this is a bug.
> 
> But the summary seems to be is that currently it is not possible to break a thread
> out of accept(2) (at least without resorting to signals).

I think you need to split the 'struct file' reference count into two different
counts similar to the how we have vref/vrele vs vhold/vdrop for vnodes.  The
fget for accept and probably most other system calls should probably be equivalent
to vhold, whereas things like open/dup (and storing an fd in a cmsg) should be
more like vref.  close() should then be a vrele().

-- 
John Baldwin