Long Format Directory Listing 15x Slower in FreeBSD 5.x
brooks at one-eyed-alien.net
Mon Sep 12 18:33:55 PDT 2005
On Mon, Sep 12, 2005 at 08:51:44PM -0400, Kris Kennaway wrote:
> On Tue, Sep 13, 2005 at 11:55:55AM +1200, Nikolai Schupbach wrote:
> > Hello,
> > We have been trying to migrate to FreeBSD 5.4 from an older 4.x release
> > for one of our busy mail servers. But we have encountered problems with
> > directory listings on 5.4.
> > Our /var/mail directory contains approximately 8,000 files doing a long
> > directory listing (ls -l) takes approximately 5 min and during this time
> > the CPU is running near 100%, on a FSBD 4 box the same directory
> > contents takes less than 20 seconds to list. Even on a directory with
> > 200 files, each file with a different owner, it will still take at least
> > 4-5 seconds to list. The problems only seems to occur when the directory
> > contains files from many different users. (as in /var/mail). If I chown
> > all the files in the /var/mail directory to a single user the directory
> > listing is near instant.
> > It appears it has something to do with 'ls' looking up the id's in the
> > password database, because if I instruct ls to display numeric IDs (ls
> > -ln), rather than converting to user and group names, the directory with
> > 8,000 files, with 8,000 different owners will list instantly.
> > The reason this concerns me so much is we are also having a problem with
> > our Washington IMAP / POP3 server on FBSD 5 using a lot of CPU while
> > operating on small and even 0 byte mailboxes, when there are approx five
> > or more concurrent POP3 or IMAP sessions. And I can't help thinking that
> > the two problems are related.
> > Does anyone have any ideas? Has anyone else noticed this problem? If I
> > can't resolve it I'm most likely going to revert to using 4.11, but I'd
> > really like to know what is going on. (and yes I'm using UFS_DIRHASH)
> Sounds like the real problem is that you have >8000 users on your
> machine and lookups are taking a long time. There has been discussion
> of this problem and how to solve it on freebsd-questions@ several
> times..I think it involves making a minor change to pwd_mkdb and
> rebuilding. Please search the archives for that mailing list.
If you are using NIS and have any compat options in /etc/nsswitch.conf
your performance will really suck in situations like this. IIRC, the
compat code is worse than O(n^2) if you look up each user and the
non-compat code is close to O(n). I'd really like to stop generating
nsswitch.conf entries that use compat in 7.0.
Any statement of the form "X is the one, true Y" is FALSE.
PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20050912/27b8fe04/attachment.bin
More information about the freebsd-stable