threads/79683: svctcp_create() fails if multiple threads call at the same time, it's not thread-safe

[Steve Sears]

>Number:         79683
>Category:       threads
>Synopsis:       svctcp_create() fails if multiple threads call at the same time, it's not thread-safe
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-threads
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Apr 08 14:30:20 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator:     Steve Sears
>Release:        5.3-STABLE, 5.4-STABLE
>Organization:
self
>Environment:
FreeBSD sjs-bsd 5.4-STABLE FreeBSD 5.4-STABLE #17: Thu Apr  7 08:15:24 EDT 2005     root at sjs-bsd:/usr/src/sys-sjs/i386/compile/SJSKERN  i386
>Description:
The svctcp_create() function returns NULL if it fails to create a
TCP/IP-based RPC service transport for the server.  On BSD, this
function often fails when several threads concurrently try to
register as servers.  When it begins to fail, subsequent retries
very rarely work (although I have seen it happen).

I frequently see svctcp_create() fail (returns NULL).
    SVCXPRT *transp =svctcp_create(sockfd, 0, 0);
with the syslog message "Could not get tcp transport" frmom the
code lib/rpc code:
static SVCXPRT * svc_com_create(fd, sendsize, recvsize, netid)
..
        if ((nconf = __rpc_getconfip(netid)) == NULL) {
                (void) syslog(LOG_ERR, "Could not get %s transport", netid);
                return (NULL);
        }

The culprit appears to be the setnetconfig(), getnetconfig(),
endnetconfig() code used by __rpc_getconfip(netid).  The
setnetconfig() returns a handle, accessed through getnetconifg(), and
terminated by endnetconfig().  This scary code attempts to share
in-memory structures and an open file pointer with multiple accessing
threads, but it appears that not all access paths guarantee that all
the data will be available for the other threads.
>How-To-Repeat:
      Don't have a small test program, sorry. Our program creates 5 listener threads, which try to invoke this code are roughly the same time.
>Fix:
      Staggering the threads by 1 second and invoking lots of retries seems to work. But it's ugly!
>Release-Note:
>Audit-Trail:
>Unformatted: