bin/60287: [patch] NSS does not handle NSS_STATUS_TRYAGAIN properly

[Lachlan O'Dea]

>Number:         60287
>Category:       bin
>Synopsis:       [patch] NSS does not handle NSS_STATUS_TRYAGAIN properly
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Dec 15 20:20:13 PST 2003
>Closed-Date:
>Last-Modified:
>Originator:     Lachlan O'Dea
>Release:        FreeBSD 5.2-BETA i386
>Organization:
Computer Associates
>Environment:
System: FreeBSD isis.ca.com 5.2-BETA FreeBSD 5.2-BETA #0: Fri Dec 5 17:00:13 EST 2003 odela01 at isis.ca.com:/usr/obj/usr/src/sys/GENERIC i386

/etc/nsswitch.conf:

passwd: files ldap
group: files ldap

>Description:
When using nss_ldap, getgrent(3) does not work properly if there are groups
in LDAP with a large number of users. When it encounters such a group,
getgrent immediately returns NULL, indicating no further groups.

The problem appears to be in /usr/src/lib/libc/net/nss_compat.c.
The __nss_compat_getgrent_r function sets a global terminate flag if
the NSS module does not return success. Once this flag is set,
subsequent calls to getgrent will return NULL.

The problem occurs if the group data is larger than the initial size
of the group buffer (1024 bytes). In this case, the NSS module returns
NSS_STATUS_TRYAGAIN, and libc will retry the call with a larger buffer.
However, the terminate flag has already been set, so the second call
goes nowhere.

The following patch will not set the terminate flag if NSS_STATUS_TRYAGAIN
is returned. It seems to work for me. Also, this code should check against
NSS_STATUS_SUCCESS, not NS_SUCCESS, although by coincidence they both have
the value 1.

>How-To-Repeat:
If you're using nss_ldap, create a group in LDAP with 100 users and
run id(1) on one of the members. The large group, and perhaps others that
come later in the search results will not be listed in the output.
Similar problems also occur with other programs that use getgrent(3).
Note that running id(1) without arguments does not show the problem.

>Fix:

--- nss_compat.c.orig	Sun Dec  7 23:58:47 2003
+++ nss_compat.c	Sun Dec  7 23:58:47 2003
@@ -144,9 +144,9 @@
 	bufsize = va_arg(ap, size_t);
 	errnop = va_arg(ap, int *);
 	status = fn(grp, buffer, bufsize, errnop);
-	if (status == NS_SUCCESS)
+	if (status == NSS_STATUS_SUCCESS)
 		*(struct group **)retval = grp;
-	else
+	else if (status != NSS_STATUS_TRYAGAIN)
 		SET_TERMINATOR(group, &terminator);
 	return (__nss_compat_result(status));
 }

>Release-Note:
>Audit-Trail:
>Unformatted: