svn commit: r191902 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Adam McDougall]

On Thu, May 07, 2009 at 08:57:06PM +0000, Kip Macy wrote:

  Author: kmacy
  Date: Thu May  7 20:57:06 2009
  New Revision: 191902
  URL: http://svn.freebsd.org/changeset/base/191902
  
  Log:
    Allow the VM to provide backpressure on the ARC cache as it does
    on Solaris.
  
  Modified:
    head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
  
  Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
  ==============================================================================
  --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c	Thu May  7 20:39:23 2009	(r191901)
  +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c	Thu May  7 20:57:06 2009	(r191902)
  @@ -131,6 +131,8 @@
   #include <sys/kstat.h>
   #include <sys/sdt.h>
   
  +#include <vm/vm_pageout.h>
  +
   static kmutex_t		arc_reclaim_thr_lock;
   static kcondvar_t	arc_reclaim_thr_cv;	/* used to signal reclaim thr */
   static uint8_t		arc_thread_exit;
  @@ -1809,6 +1811,13 @@ arc_reclaim_needed(void)
   
   #ifdef _KERNEL
   
  +	/*
  +	 * If pages are needed or we're within 2048 pages 
  +	 * of needing to page need to reclaim
  +	 */
  +	if (vm_pages_needed || (vm_paging_target() > -2048))
  +		return (1);
  +
   	if (needfree)
   		return (1);
   
This seems to cause a problem for me because it appears to put undue pressure 
on the arc when I think it shouldn't, and it seems the shrunk arc causes my 
performance to degrade severely.  I have an amd64 server with 20G of ram that 
runs ftp, cvsupd, rsyncd, two http daemons, etc and I have 
vfs.zfs.arc_max=1024M for now.  I ran "top | grep Mem" and sysctl 
kstat.zfs.misc.arcstats.size every 10 seconds shortly after the system came up 
and watched the Free column slowly drop over the next few hours.  The 
arcstats.size will generally stay quite close to 1024M.  What I see is at this 
point:

Mem: 610M Active, 16G Inact, 2226M Wired, 48M Cache, 144K Buf, 1043M Free
kstat.zfs.misc.arcstats.size: 1073922176

Free will temporarily cease to decrease while arcstats.size starts to decrease 
over the next few minutes at about the same rate Free used to decrease, until 
it hits:

Mem: 613M Active, 16G Inact, 1819M Wired, 50M Cache, 144K Buf, 1023M Free 
kstat.zfs.misc.arcstats.size: 458326208

then the arcstats.size stays around this depressed size while Free continues 
to float down towards ~650M and Wired down to ~1796M where they stabilize. 
When I revert this svn rev, arcstats.size remains around 1024M the entire 
time, and performance is fine. None of my services seem to be putting any real 
pressure on the system and it is not swapping, my novice conclusion is this 
svn change allows the lack of "Free" memory to put undue pressure on the arc 
ignoring Inact etc.  I also tried setting vfs.zfs.arc_min=950M which I think 
was helpful but perhaps not 100%, but I don't have detailed stats on that; in 
any case it would be a new workaround to add, and I'm trying hard at this 
point to be rid of workarounds so I can help test "bare" ZFS and any 
experimental patches that come up.

Thanks for your help and any input on this issue.