journaling fs and large mailbox format

[Mike Meyer]

In <433C5C8B.6000003 at FreeBSD.org>, Doug Barton <dougb at FreeBSD.org> typed:
> Mike Meyer wrote:
> > The solution isn't to avoid Maildir/mh - the solution is to tune the
> > file system for the expected usage. FreeBSD (and Unix in general)
> > gives you lots of knobs to deal with things like this. Learn to use
> > them.
> > 
> > The default block and frag size are relatively large - 2K and 16K
> > appear to be the defaults on 5.x. A quick look at my mail for 2005
> > shows 32,267 messages ranging from 280 bytes to 6+ meg, with a mean
> > size of less than 8K. I'd go with 4k blocks and a 512 byte frag size -
> > because that will give you four times as many inodes as the default
> > parameters. 8K/1K is also tempting, but you'll probably want to
> > specify -i 2048 to get the same number of inodes as you get with a
> > 4K/512b file system.
>
> In this situation, since any given file in a maildir directory is very 
> unlikely to grow, it seems to me to make sense that the right way to tune 
> the fs would be to find the median file size and make the block size large 
> enough to handle files of that size. That should give you the right tradeoff 
> between speed and efficiency.

This seems very reasonable. The trick is figuring out what "the median
file size" is. I grabbed my mail archive, but that's unlikely to be
representative of most users. You either need to guess right, or make
arrangements to reformat the file system using current dasa at regular
intervals.

Taking Doug's suggestion into account, and using the data I had, the
correct answer would be an 8K/1K file system, possibly with "-i 2048"
to double the number of inodes.

However, I did see an interesting possibility. What do you do if the
median file size is, for example, 4.1K? A 4K block won't hold your
median file. But an 8K block wastes a lot of space. You might get a
file with 0 blocks and 3 frags, assuming that UFS2 will do that, which
doesn't seem good. If UFS2 won't do that, you get a lot of half-empty
blocks, which likewise isn't good. The other option is a 4K block
size, which means you get a lot of 1 block + 1 frag files. That seems
optimal in this case.

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.