ZFS ARC under memory pressure

[Slawa Olhovchenkov]

On Thu, Aug 18, 2016 at 03:31:26PM -0500, Karl Denninger wrote:

> 
> On 8/18/2016 15:26, Slawa Olhovchenkov wrote:
> > On Thu, Aug 18, 2016 at 11:00:28PM +0300, Andriy Gapon wrote:
> >
> >> On 16/08/2016 22:34, Slawa Olhovchenkov wrote:
> >>> I see issuses with ZFS ARC inder memory pressure.
> >>> ZFS ARC size can be dramaticaly reduced, up to arc_min.
> >>>
> >>> As I see memory pressure event cause call arc_lowmem and set needfree:
> >>>
> >>> arc.c:arc_lowmem
> >>>
> >>>         needfree = btoc(arc_c >> arc_shrink_shift);
> >>>
> >>> After this, arc_available_memory return negative vaules (PAGESIZE *
> >>> (-needfree)) until needfree is zero. Independent how too much memory
> >>> freed. needfree set to 0 in arc_reclaim_thread(), when arc_size <=
> >>> arc_c. Until arc_size don't drop below arc_c (arc_c deceased at every
> >>> loop interation).
> >>>
> >>> arc_c droped to minimum value if arc_size fast enough droped.
> >>>
> >>> No control current to initial memory allocation.
> >>>
> >>> As result, I can see needless arc reclaim, from 10x to 100x times.
> >>>
> >>> Can some one check me and comment this?
> >> You might have found a real problem here, but I am short of time right now to
> >> properly analyze the issue.  I think that on illumos 'needfree' is a variable
> >> that's managed by the virtual memory system and it is akin to our
> >> vm_pageout_deficit.  But during the porting it became an artificial value and
> >> its handling might be sub-optimal.
> > As I see, totaly not optimal.
> > I am create some patch for sub-optimal handling and now test it.
> > _______________________________________________
> > freebsd-fs at freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> > To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
> 
> You might want to look at the code contained in here:
> 
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594

In may case arc.c issuse cused by revision r286625 in HEAD (and
r288562 in STABLE) -- all in 2015, not touch in 2014.

> There are some ugly interactions with the VM system you can run into if
> you're not careful; I've chased this issue before and while I haven't
> yet done the work to integrate it into 11.x (and the underlying code
> *has* changed since the 10.x patches I developed) if you wind up driving
> the VM system to evict pages to swap rather than pare back ARC you're
> probably making the wrong choice.
> 
> In addition UMA can come into the picture too and (at least previously)
> was a severe contributor to pathological behavior.

I am only do less aggresive (and more controlled) shrink of ARC size.
Now ARC just collapsed.

Pointed PR is realy BIG. I am can't read and understund all of this.
r286625 change behaivor of interaction between ARC and VM.
You problem still exist? Can you explain (in list)?

-- 
Slawa Olhovchenkov