ZFS: unlimited arc cache growth?

[Marius Nünnerich]

On Fri, Apr 17, 2009 at 16:18, Bernd Walter <ticso at cicely7.cicely.de> wrote:
> On Fri, Apr 17, 2009 at 02:50:24PM +0200, Alexander Leidinger wrote:
>> Hi,
>>
>> to fs@, please CC me, as I'm not subscribed.
>>
>> I monitored (by hand) a while the sysctls kstat.zfs.misc.arcstats.size
>> and kstat.zfs.misc.arcstats.hdr_size. Both grow way higher (at some
>> point I've seen more than 500M) than what I have configured in
>> vfs.zfs.arc_max (40M).
>
> My understanding about this is the following:
> vfs.zfs.arc_min/max are not used as min max values.
> They are used as high/low watermarks.
> If arc is more than max the arc a thread is triggered to reduce the
> arc cache until min, but in the meantime other threads can still grow
> arc so there is a race between them.

Hmm, if this is true the ARC size should go down to arc_min once it
did grow past arc_max and no new data is coming along but I do not
observe such a thing here. It simply stays near but below arc_max here
all the time. I have only /home on ZFS with moderate load.

>
>> After a while FS operations (e.g. pkgdb -F with about 900 packages...
>> my specific workload is the fixup of gnome packages after the removal
>> of the obsolete libusb port) get very slow (in my specific example I
>> let the pkgdb run several times over night and it still is not
>> finished).
>
> I've seen many workloads were prefetching can saturate disks without
> ever being used.
> You might want to try disabling prefetch.
> Of course prefetching also grows arc.
>
>> The big problem with this is, that at some point in time the machine
>> reboots (panic, page fault, page not present, during a fork1). I have
>> the impression (beware, I have a watchdog configured, as I don't know
>> if a triggered WD would cause the same panic, the following is just a
>> guess) that I run out of memory of some kind (I have 1G RAM, i386, max
>> kmem size 700M). I restarted  pkgdb several times after a reboot, and
>> it continues to process the libusb removal, but hey, this is anoying.
>
> With just 700M kmem you should set arc values extremly small and
> avoid anything which can quickly grow it.
> Unfortunately accessing many small files is a know arc filling workload.
> Activating vfs.zfs.cache_flush_disable can help speeding up arc decreasing,
> with the obvous risks of course...
>
>> Does someone see something similar to what I describe (mainly the
>> growth of the arc cache way beyond what is configured)? Anyone with
>> some ideas what to try?
>
> In my opinion the watermark mechanism can work as it is, but there should
> be a forced max - currently there is no garantied limit at all.
> Nevertheless it is up for the people which know the code to decide.
>
> --
> B.Walter <bernd at bwct.de> http://www.bwct.de
> Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
>