Need a filesystem with "unlimited" inodes

Karl Vogel vogelke+unix at
Wed Jun 10 19:06:32 UTC 2009

>> On Tue, 9 Jun 2009 03:10:46 am Matthew Seaman wrote:
M> Or store your data in a RDBMS rather than in the filesystem.

>> On Tue, 9 Jun 2009 09:45:48 -0500, Kirk Strauser <kirk at> said:
K> Hear, hear.  I'm hard pressed to imagine why you'd need 100M 1KB files.

   DBs are great when you have structured data, but semi-structured text
   (like email) makes for a very poor fit.  To see why, have a look at

   If you really need to store 100 million smallish chunks of information,
   consider using zip.  Create 256 folders named 00-ff:

       hex='0 1 2 3 4 5 6 7 8 9 a b c d e f'
       for x in $hex ; do
           for y in $hex ; do
               mkdir ${x}${y}
       exit 0

   Use the hash of your choice to map the name of each chunk to one of 256
   zipfiles under each directory.  This gives you 64k zipfiles, and if you
   put 1500 or so chunks in each one, you're pretty close to 100 million.

       me% cat mkchunks
       #!/usr/bin/perl -w
       for $chunk (@ARGV) {
           $_ = chunk2file($chunk);
           $file = "$1/$" if m/(..)(..)/;
           print "$file  $chunk\n";

       sub chunk2file {
           my $str = shift;
           my ($byte, $sum);
           use integer;
           $sum = 0;
           foreach $byte (unpack("C*", $str)) {   # SDBM hash
               $sum = $byte + 65587 * $sum;
           $sum &= 0xffff;    # keep lowest 16 bits
           no integer;
           return sprintf("%4.4x", $sum);

       me% ./mkchunks freebsd solaris
       16/  freebsd
       ca/  solaris

   You'll get a better distribution if you use a hash like Digest::SHA1.

Karl Vogel                      I don't speak for the USAF or my company

People like you are the reason people like me need medication. --bumper sticker

More information about the freebsd-questions mailing list