Long Format Directory Listing 15x Slower in FreeBSD 5.x

[Kris Kennaway]

On Mon, Sep 12, 2005 at 06:33:53PM -0700, Brooks Davis wrote:
> On Mon, Sep 12, 2005 at 08:51:44PM -0400, Kris Kennaway wrote:
> > On Tue, Sep 13, 2005 at 11:55:55AM +1200, Nikolai Schupbach wrote:
> > > Hello,
> > > 
> > > We have been trying to migrate to FreeBSD 5.4 from an older 4.x release 
> > > for one of our busy mail servers. But we have encountered problems with 
> > > directory listings on 5.4.
> > > 
> > > Our /var/mail directory contains approximately 8,000 files doing a long 
> > > directory listing (ls -l) takes approximately 5 min and during this time 
> > > the CPU is running near 100%, on a FSBD 4 box the same directory 
> > > contents takes less than 20 seconds to list. Even on a directory with 
> > > 200 files, each file with a different owner, it will still take at least 
> > > 4-5 seconds to list. The problems only seems to occur when the directory 
> > > contains files from many different users. (as in /var/mail). If I chown 
> > > all the files in the /var/mail directory to a single user the directory 
> > > listing is near instant.
> > > 
> > > It appears it has something to do with 'ls' looking up the id's in the 
> > > password database, because if I instruct ls to display numeric IDs (ls 
> > > -ln), rather than converting to user and group names, the directory with 
> > > 8,000 files, with 8,000 different owners will list instantly.
> > > 
> > > The reason this concerns me so much is we are also having a problem with 
> > > our Washington IMAP / POP3 server on FBSD 5 using a lot of CPU while 
> > > operating on small and even 0 byte mailboxes, when there are approx five 
> > > or more concurrent POP3 or IMAP sessions. And I can't help thinking that 
> > > the two problems are related.
> > > 
> > > Does anyone have any ideas? Has anyone else noticed this problem? If I 
> > > can't resolve it I'm most likely going to revert to using 4.11, but I'd 
> > > really like to know what is going on. (and yes I'm using UFS_DIRHASH)
> > 
> > Sounds like the real problem is that you have >8000 users on your
> > machine and lookups are taking a long time.  There has been discussion
> > of this problem and how to solve it on freebsd-questions@ several
> > times..I think it involves making a minor change to pwd_mkdb and
> > rebuilding.  Please search the archives for that mailing list.
> 
> If you are using NIS and have any compat options in /etc/nsswitch.conf
> your performance will really suck in situations like this.  IIRC, the
> compat code is worse than O(n^2) if you look up each user and the
> non-compat code is close to O(n).  I'd really like to stop generating
> nsswitch.conf entries that use compat in 7.0.

I'm pretty sure there's a problem with the database hash parameters
used by pwd_mkdb on systems with large numbers of users..this is what
I was alluding to above.

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20050912/666c4337/attachment.bin