NGROUPS_MAX in CURRENT - Where are those NFS problems?

Tue May 31 11:52:48 PDT 2005

On Tue, 31 May 2005, Andre Guibert de Bruet wrote:

> This question gets asked over and over on the various lists [*], but I 
> have yet to see an answer other than "We are not sure, and it is 
> probably not worth it" or "don't hike it too high, performance 
> degradation might occur":
>
> "Why doesn't CURRENT have NGROUPS_MAX set to 32 or 64?"
>
> The issue with xucred makes sense for the stable branches, but I am not 
> sure that the same compat guarantees are provided for current. I have 
> used local diffs to 64 for a number of years and have yet to run into 
> issues.
>
> I have read about fireworks with NFS, and I guess I am confuzed by why 
> this would would be the case. My "normal use" of NFS on system with 
> users in many groups never seems to have turned up any problems. The 
> following was done on an NFS client system with NGROUPS set to 64 and a 
> server with 16 in an NFS-mounted directory.

I've had a very specific request to investigate larger values of 
NGROUPS_MAX, on the order of hundreds and thousands of groups, and have 
started to do so.  Unfortunately, I have a lot of other stuff on my plate 
right now, so haven't gotten too far.  The issues I'm aware of are as 
follows:

(1) If you need a large potential increase in maximum groups, you need to
     move to a variable-length credential structure so you don't waste lots
     of kernel memory.  I have an initial prototype of this in some branch
     in Perforce.  Basically, if you ask for <16 groups, you just get
     groups in ucred, otherwise you get an extended storage allocation that
     contains a ucred.  Among other things, you now have to also maintain
     an array size.  I had hoped to get this into 6.x, but most likely it
     will go into 7.x and maybe get backported.

(2) Some kernel components are aware of, and export, group sets.  The two
     specific ones I'm aware of are the ones you've already highlighted:
     the NFS client, and xucred.  Additional credentials support was
     recently added to UNIX domain sockets, so they likely need more
     attention as well.  It's not clear to me what the right approach for
     NFS is, so we should look at what other people have done -- they most
     likely truncate the list during export.  It may be we already truncate
     in the NFS code correctly, but to be honest, I've not looked.  The
     problem, btw, will turn up when you try to access something using an
     extended group beyond the maximum supported by NFS, and because the
     client truncates the group set, access is denied (or granted).  I
     don't know if any vendors have NFS extensions to deal with this, but
     some googling is probably called for.

(3) We need to check use of group sets in user space -- particularly, for
     stack-allocated group storage, the group database code, YP, NSS, etc.
     I've not yet begun this, but suspect that if you start to support
     thousands of groups, you run into problems here.  Probably not hard to
     fix, but someone needs to do it.

If you're aware of any specific issues, please file PRs for them and point 
me at them.

Robert N M Watson

>
> %id; echo "test" > testing; sync; ls -l testing; cat testing;
> uid=1028(andy) gid=101(users) groups=101(users), 0(wheel), 102(src), 
> 106(crevil), 108(nmap), 109(osinc), 110(toast), 112(fpjet), 114(rose), 
> 115(mysql), 116(khendon), 117(ffchat), 120(cdcas), 121(masus), 122(stukings), 
> 123(uppersixth), 124(brits), 125(ncom), 126(slems), 128(architects), 
> 129(parlane), 132(bounce), 133(cryogen), 134(bikes), 135(enterprise), 
> 136(sort), 137(bcomp), 138(btestsite), 140(openj), 141(mmouse), 142(ctbm), 
> 143(realm)
> -rw-r--r-- 1 andy wheel 5 May 25 20:17 testing
> test
>
> As you can see, the output is as one would expect. I would love to provide 
> patches / testing on this "issue" but I can't seem to reproduce it.
>
> Thanks,
> Andy
>
> [*]: References:
> http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2003-12/1135.html
> http://docs.freebsd.org/cgi/getmsg.cgi?fetch=179810+0+archive/2001/freebsd-current/20011007.freebsd-current
> http://lists.freebsd.org/pipermail/freebsd-questions/2003-December/028661.html
> http://news.gw.com/freebsd.standards/5
> http://adam.kungfoohampster.com/lists/freebsd-questions/msg02574.shtml
>
> /*  Andre Guibert de Bruet  * 6f43 6564 7020 656f 2e74 4220 7469 6a20 */
> /*   Code poet / Sysadmin   * 636f 656b 2e79 5320 7379 6461 696d 2e6e */
> /*   GSM: +1 734 846 8758   * 5520 494e 2058 6c73 7565 6874 002e 0000 */
> /* WWW: siliconlandmark.com *      Tormenting bytes since 1980.       */
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
>