dirhash and dynamic memory allocation

[Jeremy Chadwick]

On Fri, Oct 14, 2011 at 02:33:18PM +0200, Miroslav Lachman wrote:
> 
> 
> Ivan Voras wrote:
> >On 14/10/2011 11:20, Miroslav Lachman wrote:
> >>Hi all,
> >>
> >>I tried some tuning of dirhash on our servers and after googlig a bit, I
> >>found an old GSoC project wiki page about Dynamic Memory Allocation for
> >>Dirhash: http://wiki.freebsd.org/DirhashDynamicMemory
> >>Is there any reason not to use it / not commit it to HEAD?
> >
> >AFAIK it's sort-of already present. In 8-stable and recent kernels you
> >can give huge amounts of memory to dirhash via vfs.ufs.dirhash_maxmem
> >(but except in really large edge cases I don't think you *need* more
> >than 32 MB), and the kernel will scale-down or free the memory if not
> >needed.
> 
> Is this change documented somewhere? Maybe it could be noticed on
> DirhashDynamicMemory wiki page. Otherwise it seems as abandoned GSoC
> project.

There is no real form of "documentation" for this kind of change, but I
do remember it being discussed on the mailing list at some point (an
announcement or something?  I forget -- man it was a while ago).

There's probably a commit message.  I can dig it up if need be, but
commit messages are often "Add Snake Wanglesnort support, I like
samgyeopsal" and not much else.  I'm not even sure what man page would
be relevant to document such pieces.  How about this?

$ sysctl -a | grep dirhash
vfs.ufs.dirhash_reclaimage: 5
vfs.ufs.dirhash_lowmemcount: 0
vfs.ufs.dirhash_docheck: 0
vfs.ufs.dirhash_mem: 1075212
vfs.ufs.dirhash_maxmem: 16777216
vfs.ufs.dirhash_minsize: 2560

$ for i in `sysctl -a | grep dirhash | cut -d: -f1`; do sysctl -d $i; done
vfs.ufs.dirhash_reclaimage: max time in seconds of hash inactivity before deletion in low VM events
vfs.ufs.dirhash_lowmemcount: number of times low memory hook called
vfs.ufs.dirhash_docheck: enable extra sanity tests
vfs.ufs.dirhash_mem: current dirhash memory usage
vfs.ufs.dirhash_maxmem: maximum allowed dirhash memory usage
vfs.ufs.dirhash_minsize: minimum directory size in bytes for which to use hashed lookup

No idea what docheck does.  But it looks like reclaimage (that's
"reclaim age") is a pretty good indicator of what Ivan is describing.

So basically, if you're worried about how much memory dirhash is taking
up, you could log/graph vfs.ufs.dirhash_mem (unit = bytes).

> >In effect, vfs.ufs.dirhash_maxmem is the upper limit - the kernel will
> >use less and will free the allocated memory in low memory situations
> >(which I've tried and it works).
> >
> >>And second question - is there any negative impact with higher
> >>vfs.ufs.dirhash_maxmem? It stil defaults to 2MB (on FreeBSD 8.2) after
> >
> >Not that I know of.
> >
> >>10 years, but I think we all are using bigger FS in these days with lot
> >>of files and directories and 2MB is not enough.
> >
> >AFAIK I've changed it to autotune so it's configured to approximately 4
> >MB on a 4 GB machine (and scales up) in 9.
> 
> I don't have 9 installed to test it. Only 8-STABLE. Hope I will test
> it soon.

The above data I showed was taken from a RELENG_8 box.  The dirhash
change in question I believe was either implemented in very early 8.x,
or very late 7.x.  Again: I can dig up the commit if requested.

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, US |
| Making life hard for others since 1977.               PGP 4BD6C0CB |