limit to number of files seen by ls?
vogelke+unix at pobox.com
Sun Jul 26 01:19:46 UTC 2009
>> On Thursday 23 July 2009 09:41:26 Karl Vogel wrote:
K> Every version of Unix I've ever used had an upper limit on the size of
K> the argument list you could pass to a program, so it won't just be "ls"
K> that's affected here. That's why I use 1,000 as a rule of thumb for the
K> maximum number of files I put in a directory.
>> On Thu, 23 Jul 2009 10:25:49 -0800,
>> Mel Flynn <mel.flynn+fbsd.questions at mailing.thruhere.net> said:
M> That arbitrary number works simply because kern.argmax default has been
M> raised somewhere in 6.x (before it was 64kB).
That arbitrary number has worked very nicely for me for 20 years under
Solaris, Linux, and several BSD variants. The main reason I stick
with 1000 is because directories are read linearly unless you're using
something like ReiserFS, and I get impatient waiting for more than that
number of filenames to be sorted when using ls.
M> And MAXNAMLEN in sys/dirent.h is 255.
That's the maximum length of a single filename in a directory. Since
I keep my filenames much shorter, I don't have a problem.
M> Knowing your way around maximum arguments length through xargs as
M> suggested in this thread is much better solution then trying to exercise
M> control over directory sizes, which may or not be under your control in
M> the first place.
Xargs is very useful, but it's not a substitute for poor design, and
it's not something you can drop into any existing pipeline without a
little thought first.
If your application is trying to create hundreds of thousands or
millions of files in any one directory, or you're creating lots of
200-character filenames from hell, then your design is a poor match for
most varieties of Unix; small directories perform better than enormous
ones, and lots of commonly-used scripts and programs will fall over
when handed zillion-file argument lists.
I'm sure the latest version of <insert-cool-OS-or-filesystem-here>
fixes all these objections, but not everyone gets to run the latest
and greatest. Don't fight your filesystem, and it won't fight you.
Karl Vogel I don't speak for the USAF or my company
Birds of a feather flock together and usually crap on your car.
More information about the freebsd-questions