UFS2 optimization for many small files

[Bill Moran]

In response to "Nguyen Tam Chinh" <unixvn at gmail.com>:
> 
> We're going to build a server with some 1Tb of over 500 million small
> files with size from 0,5k to 4k.  I'm wonder if the ufs2 can handle
> this kind of system well. From newfs(8) the min block size is 4k. This
> is not optimal in our case, a 1k or 0,5k block is more effective IMHO.
> I'd be happy if anyone can suggest what does fragment (block/8) in the
> ufs2 mean and how this parameter works. I know It's better to read the
> full ufs2 specification, but hope that someone here can give a hint.
> Please advice with optimizations or tricks.
> Thank you very much.

Read the newfs man page.

Based on your assessment of your files, I'd go with a block size of 4K
and a frag size of 500 bytes.  Blocks are broken in to frags when a file
doesn't fill an entire block.

Make sure to set -i to about 250 or so.  An inode is needed for each file
or directory on the filesystem, so you're liable to run out of inodes
with the default values.

Make sure your files are organized in a directory hierarchy.  No filesytem
that I know of performs well with huge numbers of files in a single
directory.

Please don't cross-post.  I see no reason to copy stable@ with this message.

-- 
Bill Moran
http://www.potentialtech.com