Storing a lot of little files
Eric Anderson
anderson at centtech.com
Tue Jun 15 13:53:37 GMT 2004
Eugene wrote:
>Hello freebsd-current,
>
> I need to store a lot (hundreds of millions) of very little files (from 8 bytes
> to 50K) in my filesystem.
> What's the best way to optimize it? Which newfs options can you
> recommend me?
> Can UFS2 optiomize storage of little files? For example, put some
> files in one cluster - so don't let the 8 byte file to occupy a
> whole cluster of about 16K?
> Is it possible to allocate inodes dynamically (like in ReiserFS?),
> bacause I'm not sure I can predict accuratly number of little files,
> and inodes not to finish before disk get full.
>
> Does FreeBSD support file systems except of UFS?
> What about ReiserFS, XFS, JFS?
>
>
I've never done 'hundreds of millions', but I've done about 10
million.. I can say that you will definitely need to organize them in a
heirarchical directory structure.. You'll want to look at the following
options to newfs:
-g -h -i
and these options to tunefs: -e -f -s
Eric
--
------------------------------------------------------------------
Eric Anderson Sr. Systems Administrator Centaur Technology
Talk sense to a fool and he calls you foolish.
------------------------------------------------------------------
More information about the freebsd-current
mailing list