SO_LINGER on socket with non-blocking I/O

[Chuck Swiger]

Julian Cowley wrote:
> I've been developing an application that attempts to send data from
> one host to another via TCP.  The intent is for the data transfer
> to be as reliable as possible, and to log whenever it detects
> that it has lost data (this is for a reliable syslog protocol,
> if you're wondering).  Because my application doesn't (yet) have
> application-level acknowledgments, it has to depend on TCP to make
> sure the data gets through reliably.

OK.  TCP is really good at doing what you've asked.  :-)

> When closing the socket, I want to make sure that the remaining data
> got through to the other end (or otherwise log something if it didn't).
> I've set SO_LINGER on the socket for this purpose, but one caveat is
> that I also have the socket in non-blocking mode.

When your local TCP issues a close(), the TCP stack will iterate through a 
series of steps (the FIN-WAIT stages) to ensure that any remaining data will 
be sent and acknowledged before your local machine actually releases the 
socket.  See RFC-793, "3.5.  Closing a Connection"

   CLOSE is an operation meaning "I have no more data to send."  The
   notion of closing a full-duplex connection is subject to ambiguous
   interpretation, of course, since it may not be obvious how to treat
   the receiving side of the connection.  We have chosen to treat CLOSE
   in a simplex fashion.  The user who CLOSEs may continue to RECEIVE
   until he is told that the other side has CLOSED also.  Thus, a program
   could initiate several SENDs followed by a CLOSE, and then continue to
   RECEIVE until signaled that a RECEIVE failed because the other side
   has CLOSED.  We assume that the TCP will signal a user, even if no
   RECEIVEs are outstanding, that the other side has closed, so the user
   can terminate his side gracefully.  A TCP will reliably deliver all
   buffers SENT before the connection was CLOSED so a user who expects no
   data in return need only wait to hear the connection was CLOSED
   successfully to know that all his data was received at the destination
   TCP.  Users must keep reading connections they close for sending until
   the TCP says no more data.

> My question is, what is the behavior of close() on a socket in
> non-blocking mode when SO_LINGER is set (to a non-zero time)?
> 
> There seems to be two, possibly three, possibilities according to
> some web searches I've done:
> 
> 1) the close() call immediately returns with an EWOULDBLOCK (EAGAIN)
>    error.
> 2) the call blocks anyway regardless of the non-blocking mode setting.
> 3) the call returns immediately after the connection is forcibly reset,
>    possibly losing any queued data that was to be sent.
> 
> I'm pretty sure the third possibility only happens when SO_LINGER is
> set with a linger time of 0 seconds.

Remember that the in-process reference to a socket's descriptor is not the 
same thing as the kernel's reference to the underlying TCB (or whatever 
FreeBSD calls the TCP control block).  Even if you close() the descriptor, the 
system ought to continue to process any unsent data until the TCP stack 
succeeds or times out the TCP connection.

It may be the case that what you want to use is shutdown(2), instead.

In other words, possibility #1 is probably what should happen.  #2 may happen 
if the local platform doesn't handle non-blocking I/O very well.  #3 should 
only happen if you are using a TCP stack which is broken, but some people seem 
to prefer that, so who can say?

-- 
-Chuck