Strange NFS problem implicating nfsuserd?

[Graham Allan]

On 7/1/2015 7:10 PM, Rick Macklem wrote:
>>
>> I've reproduced this across 4-5 different servers and a similar number
>> of different client systems. I'm wondering if any plausible explanation
>> suggests itself?
>>
>
> As far as I know, the domain is only set when
> the nfsuserd is started and it just uses the domain part of the machine's
> host name if not explicitly defined by "-domain". Maybe there is some bug
> in nfsuserd.c that gets tickled by the option, although I just looked and
> the argument parsing looks ok.
>
> If your xxx.yyy.zzz is identical, then I can't see how this would affect
> anything.
>
> What will cause intermittent mapping problems is having more than one
> username that maps to the same uid. (One of them will be cached at random.)
> (There was a common case of both "root" and "toor" in the password database
>   for uid == 0.)

Yes, on the face of it this report appears crazy to me too :-)

If I hadn't tried a dozen other things including reverting FreeBSD patch 
level, linux kernel/package versions, tweaking/checking ldap lookup 
settings (nslcd etc), before simply removing the "domain=" argument to 
nfsuserd, I wouldn't believe it possible. I also took a quick look 
through nfsuserd.c and couldn't see anything to explain it. I want to 
think something else must be going on, but adding or removing that 
parameter appears to toggle the problem on and off deterministically.

I was always able to get a failure within 10-60 minutes or so, so having 
the nfsuserd cache timeout at 600 minutes seems like it should eliminate 
any intermittent id lookup issues.

I guess I could try...
(1) rpcdebug on the linux client, though I'm not sure which flags to 
enable to log idmapping issues.
(2) watch nfsuserd with truss and look for different behaviors.
(3) capture NFS traffic, examine with wireshark

Graham