Re: Swap, ZFS & ARC

On Feb 2, 2023, at 6:28 AM, jbo@insane.engineer wrote:
> 
> Hello folks,
> 
> Based on a discussion on the forums not so long ago I tried to figure out how swap usage on a ZFS system plays together with ARC. However, I could find very little to no information on this which leads me to believe that there is some "core concept" I might be oblivious to.
> 
> The main question is basically this: Your system starts to swap out data from RAM to your swap partition. This swap data on disk ultimately resides somewhere in a ZFS pool. If this data then gets accessed, it might be cached by ARC essentially eating up memory again which seems counter productive.
> Is there any magic which prevents swap partitions from being loaded into ARC? Or is this a non-issue for some other reason?

I suspect this bug affects FreeBSD as well:

https://github.com/openzfs/zfs/issues/7734

From https://github.com/openzfs/zfs/issues/7734#issuecomment-422082279

I'm not an expert in this area of the code, but I think that swap on ZVOL is inherently unreliable due to writes to the swap ZVOL having to go through the normal TXG sync and ZIO write paths, which can require lots of memory allocations by design (and these memory allocations can stall due to a low memory situation). I believe this to be true for swap on ZVOL for illumos, as well as Linux, and presumably FreeBSD too (although I have no experience using it on FreeBSD, so I could be wrong).

FYI. I don't know enough about zfs internals so can't say if this poster is right or not but I too have just used a disk partition as opposed to a zvolume for swap.