zfs arc - just take it all and be good to me

Jeremy Chadwick freebsd at jdc.parodius.com
Wed Aug 11 06:17:13 UTC 2010


On Wed, Aug 11, 2010 at 06:09:10AM +0000, Marco van Tol wrote:
> On Wed, Aug 11, 2010 at 07:43:24AM +0200, Matthias Gamsjager wrote:
> > > Please, insert into /boot/loader.conf only this options:
> > > vm.kmem_size="999M"
> > > vm.kmem_size_max="999M"
> > > vfs.zfs.arc_max="160M"
> > > vfs.zfs.prefetch_disable=1
> > >
> > > And after reboot, please run dd...
> > 
> > vfs.zfs.arc_max="160M" should be enough to limit the size of the arc.
> 
> But thats the point.  I don't want to limit its size, I want it to take
> every available piece of memory it can find, while allowing applications
> to temporarely use large slabs of that same memory during their runtime.

As I described in my previous post (not sure if you've seen it yet), ZFS
on FreeBSD / the VM, at this moment in time, doesn't support this
ability.  Solaris offers this.

For the time being, you're going to have to adjust /boot/loader.conf as
described below.  That's the bottom line.

> > and I'm not sure if that  ( vm.kmem_size="999M"
> > ,vm.kmem_size_max="999M") is such a good idea.
> 
> Me neither.  I'm pretty sure I will hit the "kmem map too small"
> scenario real quick.  What is it you are testing with those settings 
> Vladislav?

You need to adjust/increase vm.kmem_size to ensure you don't experience
the "kmem map too small" condition, and make sure that vfs.zfs.arc_max
is set to something **lower** than that amount.  E.g.:

vm.kmem_size="2048M"
vfs.zfs.arc_max="1536M"

Folks need to remember that vm.kmem_size affects more than just ZFS ARC.
It's best to give yourself breathing room.

I've been harping about this problem for way too long, and complaining
in equal amount that something official needs to be written/documented
about tuning.  The FreeBSD Wiki for ZFS tuning isn't sufficient as we
all know, and as I've stated before I'm willing to write something up
but I need open communication with folks familiar with both ZFS and the
VM to help in explaining the interaction on a technical level.

-- 
| Jeremy Chadwick                                   jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |



More information about the freebsd-fs mailing list