Long Format Directory Listing 15x Slower in FreeBSD 5.x

[Nikolai Schupbach]

>>>> If you are using NIS and have any compat options in /etc/nsswitch.conf
>>>> your performance will really suck in situations like this.  IIRC, the
>>>> compat code is worse than O(n^2) if you look up each user and the
>>>> non-compat code is close to O(n).  I'd really like to stop generating
>>>> nsswitch.conf entries that use compat in 7.0.
>>>>     
>>>
>>> I'm pretty sure there's a problem with the database hash parameters
>>> used by pwd_mkdb on systems with large numbers of users..this is what
>>> I was alluding to above.
>>>   
>>
>>
>> That wouldn't suprised me.  For that matter it could be both problems.
>>
> Looks like exact problem is described here:
> http://www.freebsd.org/cgi/query-pr.cgi?pr=75855
>
> I'm not using NIS, so I will try the change /etc/nsswitch.conf  group 
> and passwd lines from 'compat' to 'files' and report back.

Last night I tried to migration over to 5.4 after making the changes in 
/etc/nsswitch.conf and the load average reduced from 50 to 0.5-1.5. This 
made a *huge* difference, however the load average is still noticeable 
higher than on FBSD 4. Using top you can see the ipop3d and imapd 
processors are using more CPU than on FBSD 4. On FBSD 4 at the same time 
(3am relatively light load) the load average is between 0.15 - 0.4.

This may not seem like a huge difference, but the hardware specs between 
the two system are huge. We are going from a P4 2.4Ghz, Raid 1 IDE 
7200RPM Drives to Dual Xeon 3.4Ghz Six Drive RAID 10 SATA 10K RPM 
Raptors /w 3ware 9000 Controller.

We have also tried increasing the hash parameters in pwd_mkdb.c, but 
this made no difference whatsoever. As a result we are going to run the 
system on FBSD 4.11 now; it seems to me that FBSD 5 has some serious 
performance issues for certain operations.

-- Nikolai.