CFT: vm_lowmem event handler patch for dirhash

[Nick Barkas]

Hello
These past few months I worked with David Malone on a Google summer of
code project for allocating memory for dirhash dynamically. The hope
is that this will allow more memory to go to dirhashes on systems that
have memory to spare, so that performance working with large
directories can be improved. We decided to actually keep the current
memory allocation scheme for dirhash unchanged, but I added a
vm_lowmem event handler (see EVENTHANDLER(9)) so that older dirhashes
will be deleted when the kernel signals that it needs more memory.
This should allow vfs.ufs.dirhash_maxmem to be increased quite a bit
above its current 2MB default.

I have patches against FreeBSD 7-STABLE (I've only tested this one on
7.0-RELEASE) here:
http://www.nada.kth.se/~barkas/dirhash_lowmem_7-stable_2008-8-14.patch
and 8-CURRENT here:
http://www.nada.kth.se/~barkas/dirhash_lowmem_head_2008-10-12.patch
Please try these out if you can, and let me know if you see any
performance benefits! I've only tested this code with some benchmark
scripts, so I am very interested to see how it does under real
workloads.

Note that the patches do not yet change the default
vfs.ufs.dirhash_maxmem because I haven't figured out what would be a
new reasonable default yet. To allow for more than 2MB of dirhashes
you'll need to set that yourself. In most of my testing on a system
with 1GB of memory, I set dirhash_maxmem to 64MB. This seems to be
more than enough space to fit, for example, dirhashes for a million
email messages in maildirs.

There are some other parameters that one could tune and potentially
achieve better performance gains. The new vfs.ufs.dirhash_reclaimage
sysctl sets the number of seconds dirhashes can remain unused before a
vm_lowmem event will unconditionally delete them. The default of 5s
works reasonably well in all my tests, although it is somewhat
workload dependent. If you change this value and see different
performance under your workload, I would definitely like to hear about
it.

For more information and a bunch of graphs with results from my
benchmarking, take a look at
http://wiki.freebsd.org/DirhashDynamicMemory. Also, I'll be giving a
talk about this project quite soon now at EuroBSDCon 2008.

Nick