Still getting kmem exhausted panic

[Andriy Gapon]

on 28/09/2010 21:13 Kostik Belousov said the following:
> On Tue, Sep 28, 2010 at 08:17:43PM +0300, Andriy Gapon wrote:
>> ARC is a ZFS private cache.
>> ZFS doesn't use unified buffer/page cache.
>> So ARC is not directly affected by pagedaemon.
>> But this is not exactly VFS layer thing.
> As a pure speculation, unbacked by any code reasing or understanding
> of the principles. Can ARC be changed to use some custom vm pager
> instead of managing memory on its own. As I understand it, ARC
> uses wired kernel mappings right now.

Yes, ARC uses malloc(9) and/or uma(9).

> If it starts using managed pages backed by a new pager, then pagedaemon
> might take actual decisions on the cache shrink by putting and reclaiming
> pages. Does ARC has some `active' count for the caching unit ? It might be
> translated to the active count for the page etc.

Well, not sure if I'd like pagedaemon directly poking ARC state.
ARC seems to be a little bit more sophisticated than pagedaemon.
One could argue that ARC is a distinctive and important feature of ZFS.
My understanding is that ARC buffer state is determined by a "group" to which it
belongs (MRU, MFU, ghost variations of those) and last access time.  With each new
access a buffer can be moved to a different group.  And when a buffer replacement
is needed, then that state plays a role in deciding which buffers to re-use.
Ditto when ARC size needs to be reduced.
Also worth noting that not always data in ARC has an associated vnode, at least
that's my understanding.

I had another crazy idea that is opposite to yours.
ARC keeps using wired pages, but there is a special pager of some kind and ARC's
pages get inserted into vnode's object when needed.  pagedaemon wouldn't manage
those pages as it does with normal ones, but instead would "hint" to ARC what is
active and what is not and then ARC would do its "smart thing".
But I think in this case we would need some way to steal pages from object when
ARC decides that it really needs to shed some pages.
This wouldn't solve a problem of double-management of memory, but at least it
could try to solve a problem of double-caching.

On the other hand, perhaps some people would find useful ZFS without ARC but with
integration into VM, if that variant retains most of features of ZFS and provides
some benefits in terms of resource usage and/or performance.

-- 
Andriy Gapon