Swap usage with ZFS

[Shane Ambler]

On 06/04/2015 00:01, Karl Denninger wrote:
> On 4/5/2015 09:23, Larry Rosenman wrote:
>> I have a -HEAD (11-CURRENT) box that has 64G of memory, but very
>> little load.
>>
>> The ZFS ARC grows to eat most of it, but I see around 200M in use in
>> SWAP.  This was under control
>> in 10.x.
>>
>> I'm wondering what information y'all need to help diagnose why.
>>
>> borg.lerctr.org /home/ler $ uname -aKU
>> FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #32 r281050:
>> Fri Apr  3 16:41:13 CDT 2015
>> root at borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER  amd64 1100067 1100067
>> borg.lerctr.org /home/ler $
>>
>>
>> borg.lerctr.org /home/ler $ top
>> last pid: 26313;  load averages:  6.92,  6.79,  6.83    up 1+16:26:05
>> 09:23:13
>> 80 processes:  4 running, 76 sleeping
>> CPU:  0.0% user, 46.9% nice,  0.3% system,  0.0% interrupt, 52.8% idle
>> Mem: 281M Active, 539M Inact, 59G Wired, 18M Cache, 8128K Buf, 1241M Free
>> ARC: 55G Total, 42G MFU, 9766M MRU, 1044K Anon, 568M Header, 3437M Other
>> Swap: 128G Total, 205M Used, 128G Free
>>
>>
> This is consistent with how the VM system is expected to behave absent
> the patches I developed for 10-STABLE (and continue to maintain for same.)
>
> In short what is going on is that ZFS (absent those patches) will allow
> ARC to grow until the pager not only wakes up and starts scavenging
> cache pages but actively starts evicting working set to the page file.
> It will then pare down the ARC but at that point you have paged out
> working set process memory.
>
> I argue this is flat-out wrong as discarding ARC instead *possibly*
> implicates one disk I/O (to retrieve said data) if the cached data is
> later needed but a page-out of RSS *always* implicates one disk I/O (to
> page out said data) and *possibly* implicates two disk operations (if
> the RSS pages are later executed.)
>
> Therefore it is /*never*/ the correct decision to favor paging out
> resident processes rather than discarding disk cache.
>
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594
>
> I do not know if this will apply against -HEAD.
>

Sounds like it's related to one of my issues. On a machine with 8G
getting 7G wired is a problem.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194654

Is there any chance that having vfs.zfs.arc_free_target smaller than
vm.v_free_target plays a part in this? or are those free targets unrelated?


-- 
FreeBSD - the place to B...Storing Data

Shane Ambler