Accessing socket APIs soon after accept

[Quattlebaum, Ryan]

Thanks, John. That's almost exactly the approach we were considering.

- Ryan Q
________________________________________
From: John Baldwin <jhb at freebsd.org>
Sent: Friday, February 27, 2015 10:20 AM
To: freebsd-net at freebsd.org
Cc: Quattlebaum, Ryan; Adrian Chadd
Subject: Re: Accessing socket APIs soon after accept

On Friday, January 16, 2015 05:07:28 PM Quattlebaum, Ryan wrote:
> Hi, Adrian. Thanks for taking a look at this.
>
> We're using FreeBSD 8.2 and httpd-2.4.10 with arp-1.5.1 and apr-util-1.5.4.
>
> The problem we're seeing is pretty intermittent, so I hope this test case
> can                                      shine a little bit of light on the
> problem. We tried debugging this on our own by adding calls to
> getsockname() right after the accept call (in
> srclib/apr/network_io/unix/sockets.c: apr_socket_accept()) and logging the
> output. That's where we saw invalid data.
>
> I took a look at the source code for the TCP syncache module and the accept
> syscall. It looks like the new child socket is available for the
> application to accept after the call to sonewconn returns, but the address
> information isn't set until further down in the function. Wouldn't this
> open a window where an application could accept on a socket that the
> syncache code isn't done configuring?

This is a bug in 8.x it seems.  It was fixed in HEAD in this commit:

------------------------------------------------------------------------
r261242 | gnn | 2014-01-28 15:28:32 -0500 (Tue, 28 Jan 2014) | 10 lines

Decrease lock contention within the TCP accept case by removing
the INP_INFO lock from tcp_usr_accept.  As the PR/patch states
this was following the advice already in the code.
See the PR below for a full disucssion of this change and its
measured effects.

PR:             183659
Submitted by:   Julian Charbon
Reviewed by:    jhb

In particular, that commit changed the syncache code to not place the socket
in the queue until the end of the function via soisconnected().

You can probably merge the tcp_syncache.c portion of that change back to 8.x
without any ill effects and it should fix your problem.

--
John Baldwin