dirhash and dynamic memory allocation

[Jeremy Chadwick]

On Fri, Oct 21, 2011 at 07:16:27PM -0700, Doug Barton wrote:
> On 10/21/2011 08:38, Miroslav Lachman wrote:
> > Hi, I am back on this topic...
> > 
> > Ivan Voras wrote:
> >> On 14/10/2011 11:20, Miroslav Lachman wrote:
> >>> Hi all,
> >>>
> >>> I tried some tuning of dirhash on our servers and after googlig a bit, I
> >>> found an old GSoC project wiki page about Dynamic Memory Allocation for
> >>> Dirhash: http://wiki.freebsd.org/DirhashDynamicMemory
> >>> Is there any reason not to use it / not commit it to HEAD?
> >>
> >> AFAIK it's sort-of already present. In 8-stable and recent kernels you
> >> can give huge amounts of memory to dirhash via vfs.ufs.dirhash_maxmem
> >> (but except in really large edge cases I don't think you *need* more
> >> than 32 MB), and the kernel will scale-down or free the memory if not
> >> needed.
> >>
> >> In effect, vfs.ufs.dirhash_maxmem is the upper limit - the kernel will
> >> use less and will free the allocated memory in low memory situations
> >> (which I've tried and it works).
> > 
> > So the current behavior is that on 7.3+ and 8.x we have smaller average
> > dirhash buffer (by default) than it was initialy 10 years ago. Because
> > it starts as 2MB fixed size and now we have 2MB max, which is lowered in
> > low mem situations... and sometimes it is set to 0MB!
> > 
> > I caught this 2 days ago:
> > 
> > root at rip ~/# sysctl vfs.ufs
> > vfs.ufs.dirhash_reclaimage: 5
> > vfs.ufs.dirhash_lowmemcount: 36953
> > vfs.ufs.dirhash_docheck: 0
> > vfs.ufs.dirhash_mem: 0
> > vfs.ufs.dirhash_maxmem: 8388608
> > vfs.ufs.dirhash_minsize: 2560
> > 
> > I set maxmem to 8MB in sysctl.conf to increase performance and
> > dirhash_mem 0 is really bad surprise!
> > 
> > I am worrying about bad performance in situation where dirhash is
> > emptied in situations, where server is already running at maximum
> > performance (there is some memory hungry process and system can start
> > swapping to disk + dirhash is efectively disabled)
> > 
> > I found a PR kern/145246
> > http://www.freebsd.org/cgi/query-pr.cgi?pr=145246
> > 
> > Is it possible to add some dirhash_minmem limit to not clear all the
> > dirhash memory?
> > So I can set dirhash_minmem=2MB dirhash_maxmem=16MB and then dirhash_mem
> > will be allways between these two limits?
> 
> Isn't that what vfs.ufs.dirhash_minsize is for? I think given that there
> is a lot more memory in modern systems setting that higher by default is
> probably a good idea. Or maybe I'm misunderstanding what that knob does?

I believe the function of that sysctl is different.  It's not the
"minimum amount of dirhash memory to retain", it's:

$ sysctl -d vfs.ufs.dirhash_minsize
vfs.ufs.dirhash_minsize: minimum directory size in bytes for which to use hashed lookup

The sysctl should really be named "dirhash_mindirsize".

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, US |
| Making life hard for others since 1977.               PGP 4BD6C0CB |