zfs arc - just take it all and be good to me

[Marco van Tol]

On Tue, Aug 10, 2010 at 06:49:19PM -0700, Jeremy Chadwick wrote:
> On Tue, Aug 10, 2010 at 09:44:18PM +0000, Marco van Tol wrote:
> > [...]
> > 
> > All in all this looks like a close attempt at zfs memory being auto
> > tuned while using maximum amount of memory.  The only problem is, nobody
> > else is doing it like this so its very likely that this is not the smart
> > thing to do.
> 
> I'm not sure what "nobody else is doing it like this" means, but Solaris
> 10 behaves exactly as you describe -- the ARC takes up as much memory as
> it can.  When an application or other piece of the kernel wants memory
> which the ARC can free up, it releases memory.  There's no tuning
> required on Solaris; it "just works".
> 
> At my day job (where we use Solaris 10), when we introduced ZFS into the
> picture, many of our memory usage monitors began firing indicating lack
> of free memory due to ARC usage.  We had to add some code to our
> check_system_memory monitor which called kstat to examine the
> zfs:0:arcstats:size and zfs:0:arcstats:c_min properties and then add
> that to the overall amount of memory available.  Code in question:
> 
> my $zfsarcstatssize = "zfs:0:arcstats:size";
> my $zfsarcstatsmin = "zfs:0:arcstats:c_min";
> my $zfsarcsize;
> my $zfsarcmin = 0;
> if ($OS eq 'solaris' and -x $kstat) {
>     my @dump = `$kstat -p $zfsarcstatssize $zfsarcstatsmin`;
>     unless ($?) {
>         foreach my $dump (@dump) {
>             chomp $dump;
>             my ($label, $size) = split(/\t/, $dump);
>             if ($label eq $zfsarcstatssize) {
>                 $zfsarcsize = sprintf("%.2f", $size/1024/1024);
>             }
>             if ($label eq $zfsarcstatsmin) {
>                 $zfsarcmin = sprintf("%.2f", $size/1024/1024);
>             }
>         }
>         $real_available += $zfsarcsize - $zfsarcmin if (defined $zfsarcsize and $zfsarcsize > $zfsarcmin);
>     }
> }
> 
> I believe OpenSolaris behaves the same as Solaris 10 in this regard.
> Their VM model differs from that of FreeBSD.
> 
> I've talked about this in the past (see paragraph starting with "Fast
> forward to today").  I can dig up John's original response (talking
> about how the VM needs to be improved in FreeBSD to accomplish the
> equivalent of what Solaris does) if need be.
> 
> http://lists.freebsd.org/pipermail/freebsd-fs/2010-May/008598.html

Thank you for your reply.  I have to go to work now first, but I will
read more closely what you're writing later today.

One thing I realized by the way is that I messed something up in the
test 4 "run it all together" scenario.  I wrote I used value
"1_000_000_000", but that will cause a perl process that is too large.
Use 500_000_000 instead and what I wrote happens like I wrote it,
together with strongly fluctuation arc sizes as it runs.  As expected
and desired.

By the way, uname -a output for the machine running the tests is (line
wrapped):

FreeBSD hostname.tld 8.1-PRERELEASE FreeBSD 8.1-PRERELEASE #3: Wed
Jul 28 22:38:28 CEST 2010
root at hostname.tld:/usr/obj/usr/src/sys/MAGICA  amd64

The RELENG tag is: RELENG_8

Thanks!

Marco

-- 
Als de redding het hoogst is, is de nood nabij!