svn commit: r319722 - in head: sys/cam/ctl sys/dev/iscsi sys/kern sys/netgraph sys/netgraph/bluetooth/socket sys/netinet sys/ofed/drivers/infiniband/core sys/ofed/drivers/infiniband/ulp/sdp sys/rpc...

[Gleb Smirnoff]

  Conrad,

  this should be fixed by r320324. Sorry for inconvenience.

On Fri, Jun 23, 2017 at 09:48:24PM -0700, Conrad Meyer wrote:
C> Hi Gleb,
C> 
C> We suspect this revision has broken setsockopt(SO_SNDBUF), etc., on
C> listen sockets, as used by e.g. nginx.
C> 
C> Example backtrace: http://imgur.com/a/fj5JQ
C> 
C> The proposed mechanism is the destroyed snd/rcv sockbufs (and
C> associated locks) as part of solisten_proto().
C> 
C> Best,
C> Conrad
C> 
C> 
C> 
C> On Thu, Jun 8, 2017 at 2:30 PM, Gleb Smirnoff <glebius at freebsd.org> wrote:
C> > Author: glebius
C> > Date: Thu Jun  8 21:30:34 2017
C> > New Revision: 319722
C> > URL: https://svnweb.freebsd.org/changeset/base/319722
C> >
C> > Log:
C> >   Listening sockets improvements.
C> >
C> >   o Separate fields of struct socket that belong to listening from
C> >     fields that belong to normal dataflow, and unionize them.  This
C> >     shrinks the structure a bit.
C> >     - Take out selinfo's from the socket buffers into the socket. The
C> >       first reason is to support braindamaged scenario when a socket is
C> >       added to kevent(2) and then listen(2) is cast on it. The second
C> >       reason is that there is future plan to make socket buffers pluggable,
C> >       so that for a dataflow socket a socket buffer can be changed, and
C> >       in this case we also want to keep same selinfos through the lifetime
C> >       of a socket.
C> >     - Remove struct struct so_accf. Since now listening stuff no longer
C> >       affects struct socket size, just move its fields into listening part
C> >       of the union.
C> >     - Provide sol_upcall field and enforce that so_upcall_set() may be called
C> >       only on a dataflow socket, which has buffers, and for listening sockets
C> >       provide solisten_upcall_set().
C> >
C> >   o Remove ACCEPT_LOCK() global.
C> >     - Add a mutex to socket, to be used instead of socket buffer lock to lock
C> >       fields of struct socket that don't belong to a socket buffer.
C> >     - Allow to acquire two socket locks, but the first one must belong to a
C> >       listening socket.
C> >     - Make soref()/sorele() to use atomic(9).  This allows in some situations
C> >       to do soref() without owning socket lock.  There is place for improvement
C> >       here, it is possible to make sorele() also to lock optionally.
C> >     - Most protocols aren't touched by this change, except UNIX local sockets.
C> >       See below for more information.
C> >
C> >   o Reduce copy-and-paste in kernel modules that accept connections from
C> >     listening sockets: provide function solisten_dequeue(), and use it in
C> >     the following modules: ctl(4), iscsi(4), ng_btsocket(4), ng_ksocket(4),
C> >     infiniband, rpc.
C> >
C> >   o UNIX local sockets.
C> >     - Removal of ACCEPT_LOCK() global uncovered several races in the UNIX
C> >       local sockets.  Most races exist around spawning a new socket, when we
C> >       are connecting to a local listening socket.  To cover them, we need to
C> >       hold locks on both PCBs when spawning a third one.  This means holding
C> >       them across sonewconn().  This creates a LOR between pcb locks and
C> >       unp_list_lock.
C> >     - To fix the new LOR, abandon the global unp_list_lock in favor of global
C> >       unp_link_lock.  Indeed, separating these two locks didn't provide us any
C> >       extra parralelism in the UNIX sockets.
C> >     - Now call into uipc_attach() may happen with unp_link_lock hold if, we
C> >       are accepting, or without unp_link_lock in case if we are just creating
C> >       a socket.
C> >     - Another problem in UNIX sockets is that uipc_close() basicly did nothing
C> >       for a listening socket.  The vnode remained opened for connections.  This
C> >       is fixed by removing vnode in uipc_close().  Maybe the right way would be
C> >       to do it for all sockets (not only listening), simply move the vnode
C> >       teardown from uipc_detach() to uipc_close()?
C> >
C> >   Sponsored by:         Netflix
C> >   Differential Revision:        https://reviews.freebsd.org/D9770
C> >
C> > Modified:
C> >   head/sys/cam/ctl/ctl_ha.c
C> >   head/sys/dev/iscsi/icl_soft_proxy.c
C> >   head/sys/kern/sys_socket.c
C> >   head/sys/kern/uipc_accf.c
C> >   head/sys/kern/uipc_debug.c
C> >   head/sys/kern/uipc_sockbuf.c
C> >   head/sys/kern/uipc_socket.c
C> >   head/sys/kern/uipc_syscalls.c
C> >   head/sys/kern/uipc_usrreq.c
C> >   head/sys/netgraph/bluetooth/socket/ng_btsocket_l2cap.c
C> >   head/sys/netgraph/bluetooth/socket/ng_btsocket_rfcomm.c
C> >   head/sys/netgraph/bluetooth/socket/ng_btsocket_sco.c
C> >   head/sys/netgraph/ng_ksocket.c
C> >   head/sys/netinet/sctp_input.c
C> >   head/sys/netinet/sctp_syscalls.c
C> >   head/sys/netinet/sctp_sysctl.c
C> >   head/sys/netinet/sctp_usrreq.c
C> >   head/sys/netinet/tcp_subr.c
C> >   head/sys/netinet/tcp_syncache.c
C> >   head/sys/netinet/tcp_timewait.c
C> >   head/sys/ofed/drivers/infiniband/core/iwcm.c
C> >   head/sys/ofed/drivers/infiniband/ulp/sdp/sdp_main.c
C> >   head/sys/rpc/svc_vc.c
C> >   head/sys/sys/sockbuf.h
C> >   head/sys/sys/socket.h
C> >   head/sys/sys/socketvar.h
C> >   head/usr.bin/netstat/inet.c

-- 
Totus tuus, Glebius.