pthread_setugid_np

[Robert Watson]

On Wed, 27 May 2009, Zachary Loafman wrote:

> Isilon has need of per-thread impersonation. We're looking at implementing 
> something like the pthread_setugid_np mechanism found on OS X, loosely 
> documented in the code:
>
> http://fxr.watson.org/fxr/source/bsd/kern/kern_prot.c?v=xnu-1228 (see settid 
> and setgroups1)
>
> and some here: 
> http://lists.apple.com/archives/perfoptimization-dev/2008/Jan/msg00043.html
>
> Does anyone have strong objections to Apple's APIs here? There's obviously 
> no portable itnerface to handle it, and it seems a little saner to just 
> adopt someone else's API/semantics rather than reinvent.

I'm not opposed to adding APIs along these lines, as long as it is done Very 
Carefully(tm).  Some experience here suggests these sorts of things are very 
easy to do wrong, anyway. :-)

Having spent some time investigating and using the APIs on Mac OS X in the 
last year, I can report that their usage is at times inconsistent. 
Applications frequently fail to properly update their full thread credentials, 
assuming that updating only the euid or egid is sufficient, perhaps neglecting 
other IDs or additional groups.  While this is definitely an application bug, 
it is also relevant to the base OS because we do provide a set of 
credential-management library functions (especially setlogincontext(3)) that 
will not be aware of thread credentials.

Per-thread credentials also require semantics that effectively preclude M:N 
threading with usersapce context switching being used in the future (or, at 
least, requires user threads with different credentials to use different 
kernel-visible threads, or the addition of explicit ucred descriptors to allow 
credential context to be saved and restored), which while not currently a huge 
concern, is worth thinking about.

There are also potential concerns about other credential elements, such as MAC 
labels provided by policies that assume timely update of the label across all 
threads (i.e., on next entry to the kernel) as part of their semantics, and 
might not respond well to individual threads having other labels.  This might 
be addressed by MAC policies having the opportunity to force an update to the 
per-thread credential even when running in per-thread mode in order to 
propagate their own changes, but we'd have to think a bit about the specific 
requirements.

Finally: one of the things Apple found with lots of use of daemons that either 
switched credentials a lot in order to impersonate many users out of a singlle 
process, is that they ended up with a lot more different credentials in use at 
a time, as the fleeting credentials get referenced for the long-term by file 
descriptors opened when the credential was active.  Our reference counting 
model is intended to save memory in the case where lots of credentials are the 
same but changes are infrequent, and so you can see kernel memory use balloon. 
The per-thread case is a bit better behaved than the simple per-process case 
frequently switching, but it's worth watching out for this.  Apple addressed 
the problem by doing a coalesce stage after creating and initializing a new 
credential, in which potential existing credentials with identical contents 
are searched for and then used instead, discarding the new one, which comes 
with some overhead.

Robert N M Watson
Computer Laboratory
University of Cambridge