svn commit: r313043 - head/sys/kern

[Gleb Smirnoff]

  Hartmut,

  Thanks for explanation! Is there a test program available to reproduce the problem? I want
to try the sequence on my branch.

On Thu, Feb 02, 2017 at 08:29:20AM +0000, Hartmut.Brandt at dlr.de wrote:
H> To be honest - I feared that when I saw your messages regarding this. Here is my original message from july. Attached is also a small test program.
H> 
H> Hi,
H> 
H> I'm trying to use asio (that's boost::asio without boost) to handle listening sockets asynchronuosly. This appears not to work. There are also some reports on the net about this problem. I was able to reproduce the problem with a small C-programm that does the same steps as asio. The relevant sequence of system calls is:
H> 
H> kqueue()					 = 3 (0x3)
H> socket(PF_INET,SOCK_STREAM,6)			 = 4 (0x4)
H> setsockopt(0x4,0xffff,0x800,0x7fffffffea2c,0x4)	 = 0 (0x0)
H> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0)
H> setsockopt(0x4,0xffff,0x4,0x7fffffffea2c,0x4)	 = 0 (0x0)
H> bind(4,{ AF_INET 0.0.0.0:8080 },16)		 = 0 (0x0)
H> listen(0x4,0x80)				 = 0 (0x0)
H> ioctl(4,FIONBIO,0xffffea2c)			 = 0 (0x0)
H> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0)
H> kevent(3,0x0,0,0x7fffffffe5a0,32,0x0)		 ERR#4 'Interrupted system call'
H> 
H> The problem here is that asio registers each file descriptor with EVFILT_READ and EVFILT_WRITE as soon as it is opened (first kevent call). 
H> After bringing the socket into the listening state and when async_accept() is called it registers the socket a second time. According to the man page this is perfectly legal and can be used to modify the registration.
H> 
H> With this sequence of calls kevent() does not return when a connection is established successfully.
H> 
H> I tracked down the problem and the reason is in soo_kqfilter(). This is called for the first EVFILT_READ registration and decides based on the SO_ACCEPTCONN flag which filter operations to use solisten_filtops or soread_filtops. In this case it chooses soread_filtops.
H> 
H> The second EVFILT_READ registration does not call soo_kqfilter() again, but just updates the filter from the data and fflags field so the listening socket ends up with the wrong filter operations.
H> 
H> 
H> 
H> -----Original Message-----
H> From: Gleb Smirnoff [mailto:glebius at FreeBSD.org] 
H> Sent: Wednesday, February 01, 2017 7:08 PM
H> To: Hartmut Brandt
H> Cc: src-committers at freebsd.org; svn-src-all at freebsd.org; svn-src-head at freebsd.org
H> Subject: Re: svn commit: r313043 - head/sys/kern
H> 
H> On Wed, Feb 01, 2017 at 01:12:07PM +0000, Hartmut Brandt wrote:
H> H> Author: harti
H> H> Date: Wed Feb  1 13:12:07 2017
H> H> New Revision: 313043
H> H> URL: https://svnweb.freebsd.org/changeset/base/313043
H> H> 
H> H> Log:
H> H>   Merge filt_soread and filt_solisten and decide what to do when checking
H> H>   for EVFILT_READ at the point of the check not when the event is registers.
H> H>   This fixes a problem with asio when accepting a connection.
H> H>   
H> H>   Reviewed by:	kib@, Scott Mitchell
H> 
H> This goes into opposite direction with what I am doing:
H> 
H> https://reviews.freebsd.org/D9356
H> 
H> Can you please explain the problem with asio when accepting a connection?
H> 
H> -- 
H> Totus tuus, Glebius.
H> 

H> #include <sys/socket.h>
H> #include <sys/types.h>
H> #include <sys/event.h>
H> #include <sys/filio.h>
H> 
H> #include <sys/ioctl.h>
H> #include <netinet/in.h>
H> #include <stdio.h>
H> #include <string.h>
H> #include <err.h>
H> 
H> static void
H> wait_loop(int kq, int sock)
H> {
H> 	struct kevent ev[32];
H> 	struct sockaddr_in addr;
H> 	socklen_t socklen;
H> 
H> 	for (;;) {
H> 		int nev = kevent(kq, NULL, 0, ev, 32, NULL);
H> 		if (nev < 1)
H> 			err(1, "kevent");
H> 		for (int i = 0; i < nev; ++i) {
H> 			if (ev[i].ident == sock) {
H> 				printf("accept\n");
H> 				int fd = accept(ev[i].ident,
H> 				    (struct sockaddr *)&addr, &socklen);
H> 				if (fd == -1)
H> 					err(1, "accept");
H> 			}
H> 		}
H> 	}
H> }
H> 
H> int
H> main()
H> {
H> 	struct sockaddr_in addr;
H> 
H> 	/* open a TCP socket */
H> 	int kq = kqueue();
H> 
H> 	int sock = socket(PF_INET, SOCK_STREAM, 0);
H> 
H> 	struct kevent ev[2];
H> 	EV_SET(&ev[0], sock, EVFILT_READ, EV_ADD | EV_CLEAR, 0, 0, NULL);
H> 	EV_SET(&ev[1], sock, EVFILT_WRITE, EV_ADD | EV_CLEAR, 0, 0, NULL);
H> 
H> 	int opt = 1;
H> 	setsockopt(sock, SOL_SOCKET, SO_NOSIGPIPE, &opt, sizeof(opt));
H> 
H> 	if (kevent(kq, ev, 2, NULL, 0, NULL) == -1)
H> 	    err(1, "kevent");
H> 
H> 	setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
H> 
H> 	memset(&addr, 0, sizeof(addr));
H> 	addr.sin_port = htons(10000);
H> 
H> 	bind(sock, (struct sockaddr *)&addr, sizeof(addr));
H> 	listen(sock, 0x80);
H> 
H> 	ioctl(sock, FIONBIO, &opt);
H> 
H> 	if (kevent(kq, ev, 2, NULL, 0, NULL) == -1)
H> 		err(1, "kevent");
H> 
H> 	wait_loop(kq, sock);
H> }


-- 
Totus tuus, Glebius.