UFS2 limits

[Ian]

On Sun, 9 Nov 2008 13:10:46 Jeremy Chadwick wrote:
> On Sun, Nov 09, 2008 at 01:40:51AM +0000, no-spam at people.net.au wrote:
> > Hi,
> > I have a FreeBSD server that has about 10,500 subdirectories within a
> > single directory.
> > This number will keep rising and I assume UFS2 has a limit to the number
> > of sub-directories in a single directory - can anyone tell me what it is?
>
> As far as I know, there is no such limit on the number of files/dirs
> inside of a directory.
>
> I don't want to change the topic of discussion, but I *highly* recommend
> you ***stop*** whatever it is you're doing that is creating such a
> directory structure.  Software which has to iterate through that
> directory using opendir() and readdir() will get slower and slower as
> time goes on.
>
> If this is something you've written or have control over or can work
> with engineers in regards to, I recommend you change your directory
> naming scheme to have separate subdirectories with the first 2 or 3
> letters of the directory you wish to create.  E.g.:
>
> /some/place/00/00ilikezeros/*
> /some/place/01/01binaryheaven/*
> /some/place/aa/aardvarks/*
> /some/place/ab/abuse/*
> /some/place/ac/actuary/*
> ...
> /some/place/xy/xylophones/*
>
> You get the point.
>
> Traversing this structure is much more efficient, and requires very
> little code change on your part.  Those who run nameservers that host
> many zones, for example, use this structure to ensure the daemon doesn't
> take 32498231 years to start up.
>
> > What about ZFS?
> >
> > At some point I'll have to re-arrange things so that I have a deeper
> > directory structure, just wondering when I'll hit the limit so I can plan
> > in advance :-)
>
> What baffles me is why you're looking at this problem from a " "how can
> the filesystem solve this engineering mistake I made for me" standpoint,
> rather than "how can I solve this engineering mistake I made so that it
> doesn't impact the filesystem".  Very strange.  Sometimes looking at
> things in a different light makes all the difference.
>
> Hope this helps.
>
> P.S. -- I hope this mail makes it to you, because your From line is
> no-spam at people.net.au (I'll be surprised if your account name really is
> that!).


Thanks for that Jeremy. I didn't invent this structure, but I daresay I can 
either modify it or get the original writer to do that. I never really gave 
it a thought before now - it was the system I was given to work with and it's 
worked fine so far, except when I try to list the contents of the directory - 
that takes ages!
All the folders are 7 digit numbers and we are up to approx. 0010500 entries 
(ie subdirs) so far. 
I guess it will just be a matter of experimenting to find the optimum number 
of sub-sub-directories per sub-directory :-/

Oh, and yes, that email address does work - I use it for mailing lists and 
other stuff where I'm likely to get spammed - it's ironic really :-)

Cheers,
-- 
Ian
gpg key: http://home.swiftdsl.com.au/~imoore/no-spam.asc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20081109/9d1004c0/attachment.pgp