Storing a lot of little files
julian at elischer.org
Tue Jun 15 23:10:38 GMT 2004
On Tue, 15 Jun 2004, Cyrille Lefevre wrote:
> "Eugene" <el2000 at km.ru> wrote:
> > Hello freebsd-current,
> > I need to store a lot (hundreds of millions) of very little files (from 8
> > to 50K) in my filesystem.
> some times ago, there where something called "inode fs" (aka IFS).
> unfortunatelly, this was killed from -current (5.x) two years ago.
> more details here (google "freebsd +ifs +inode"):
It's "supposed" to be coming back some time..
there are also "microfiles" (< 64 bytes)
which store the data in the inode.. You can simulate this
by using a symlink and using readlink(2) to instead of read(2)
when ever you encounter a symlink. (if you have source).
There were patches for microfiles floating around somewhere but
they were "pre-softupdates". I still think it would be a good idea.
/etc/malloc.conf uses this approach.
> > What's the best way to optimize it? Which newfs options can you
> > recommend me?
> so, the best way would be to have multi-level directories to reduce
> the number of entries in one directory whatever the underlying file
> system is (except, maybe, database-like filesystems).
> something like :
> using google "million +files +directory +fs" :
> Cyrille Lefevre.
> home: mailto:cyrille.lefevre at laposte.net
> freebsd-current at freebsd.org mailing list
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
More information about the freebsd-current