Please explain FreeBSD 8.0/ZFS memory usage patterns

Benjamin Lutz mail at maxlor.com
Tue Jan 12 10:06:28 UTC 2010


Hello,

I've set up a FreeBSD-8.0/amd64 system with ZFS as a home server, and I'm a 
bit puzzled by it's memory usage pattern; I'm seeing total memory usage 
oscillate between roughly 50% and 90% of my RAM. So I've tracked memory 
usage over the course of about 12 hours and graphed it here:

http://www.maxlor.com/temp/mem.png

The unit is MiB;
kern = vm.kmem_size,
proc = sum of RSS of all processes (since some memory will be shared between 
processes, this is an upper bound),
active = vm.stats.vm.v_active_count,
wire = vm.stats.vm.v_wire_count,
arc = kstat.zfs.misc.arcstats.size

A few other details about the machine in question:
avail memory = 3581587456 (3415 MiB)
kstat.zfs.misc.arcstats.c_min: 93800960 (89 MiB)
kstat.zfs.misc.arcstats.c_max: 750407680 (716 MiB)

The machine runs many services (mostly mail, filesharing via NFS, apache, 
squeezebox server, rtorrent), but is generally very lightly loaded (as I 
said, it's a home server with mostly 1 user). As you can see from the graph 
and numbers, memory pressure seems to be about nonexistant. There is a root 
UFS file system, but it sees little use; /home, /usr and /var are on ZFS.

Now, I have a couple of questions:

After subtracting the ARC size from the wired memory size, there are still 
about 500 MiB of wired memory left that I can't account for. This is much 
higher than I'm used to seeing on FreeBSD 7.2/i386 systems without ZFS, 
where the wired amount is typically in the 50 MiB range. What are those 500 
MiB used for, or how do I find out?

Same goes for active memory: if I subtract the "proc" amount from 
the "active" amount, there are 500 MiB left over (probably more, since proc 
is an upper bound) which I presume are used by the kernel... but what for? 
Does ZFS require that much memory for things other than the ARC?

In the first half of the graph, there's a torrent download running, which I 
assume causes the large fluctuations in memory usage. However, memory usage 
of the torrent program is pretty much constant while it runs; is FreeBSD 
simply behaving lazily here, not bothering to free memory as long as there 
isn't any memory pressure?

Any insight into why the numbers are as they are is appreciated.

Cheers
Benjamin


More information about the freebsd-stable mailing list