ZFS: unlimited arc cache growth?
Alexander at Leidinger.net
Sat Apr 18 07:39:14 UTC 2009
On Fri, 17 Apr 2009 16:18:17 +0200 Bernd Walter
<ticso at cicely7.cicely.de> wrote:
> On Fri, Apr 17, 2009 at 02:50:24PM +0200, Alexander Leidinger wrote:
> > Hi,
> > to fs@, please CC me, as I'm not subscribed.
> > I monitored (by hand) a while the sysctls
> > kstat.zfs.misc.arcstats.size and kstat.zfs.misc.arcstats.hdr_size.
> > Both grow way higher (at some point I've seen more than 500M) than
> > what I have configured in vfs.zfs.arc_max (40M).
> My understanding about this is the following:
> vfs.zfs.arc_min/max are not used as min max values.
> They are used as high/low watermarks.
> If arc is more than max the arc a thread is triggered to reduce the
> arc cache until min, but in the meantime other threads can still grow
> arc so there is a race between them.
500M (more than 10 times my max) after a night seems to be a big race...
> > After a while FS operations (e.g. pkgdb -F with about 900
> > packages... my specific workload is the fixup of gnome packages
> > after the removal of the obsolete libusb port) get very slow (in my
> > specific example I let the pkgdb run several times over night and
> > it still is not finished).
> I've seen many workloads were prefetching can saturate disks without
> ever being used.
> You might want to try disabling prefetch.
> Of course prefetching also grows arc.
Prefetching is already disabled in this case.
> > The big problem with this is, that at some point in time the
> > machine reboots (panic, page fault, page not present, during a
> > fork1). I have the impression (beware, I have a watchdog
> > configured, as I don't know if a triggered WD would cause the same
> > panic, the following is just a guess) that I run out of memory of
> > some kind (I have 1G RAM, i386, max kmem size 700M). I restarted
> > pkgdb several times after a reboot, and it continues to process the
> > libusb removal, but hey, this is anoying.
> With just 700M kmem you should set arc values extremly small and
> avoid anything which can quickly grow it.
> Unfortunately accessing many small files is a know arc filling
> workload. Activating vfs.zfs.cache_flush_disable can help speeding up
> arc decreasing, with the obvous risks of course...
I have this:
vfs.zfs.vdev.cache.bshift="13" # device read ahead: 8k
vfs.zfs.vdev.max_pending="6" # congruent request to the device, + for NCQ
More information about the freebsd-current