UFS2 optimization for many small files

[Nguyen Tam Chinh]

On 7/2/07, Nikolay Pavlov <quetzal at zone3000.net> wrote:
> On Wednesday, 27 June 2007 at 14:11:19 +0400, Nguyen Tam Chinh wrote:
> > Greetings,
> >
> > We're going to build a server with some 1Tb of over 500 million small
> > files with size from 0,5k to 4k.  I'm wonder if the ufs2 can handle
> > this kind of system well. From newfs(8) the min block size is 4k. This
> > is not optimal in our case, a 1k or 0,5k block is more effective IMHO.
> > I'd be happy if anyone can suggest what does fragment (block/8) in the
> > ufs2 mean and how this parameter works. I know It's better to read the
> > full ufs2 specification, but hope that someone here can give a hint.
> > Please advice with optimizations or tricks.
> > Thank you very much.
> >
> > --
> > With best regards,        |         The Power to Serve
> > Nguyen Tam Chinh      |      http://www.FreeBSD.org
> > _______________________________________________
> > freebsd-stable at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>
> I am not aware of any ZFS results on such tasks, may be you will be the
> one who share them ;) However RaiserFS whould be the best choise on such
> spesific case. It's not available on FreeBSD currently.
> I don't think UFS can handle a huge amount of small files effectively.
> Of course gjournal could be an option for fsck problems, but how do you
> plan to backup or sync this storage?
>

I'm aware of the fsck/backup problems. In our case there's no need for
backup so i went with ufs2. The current configuration is 4x250Gb disks
with bloc/frag ratio 4k/512b. We're generating files with the average
size of 6k ('cause the compress procedure does not work as well as we
estimated).
After a week I think we could collect some statistics in production.
Anyway, in this case a 8k/1k would be more effective for us. Hope that
I can test this in the next server.

-- 
With best regards,        |         The Power to Serve
Nguyen Tam Chinh      |      http://www.FreeBSD.org