RPI3 swap experiments ["was killed: out of swap space" with: "v_free_count: 5439, v_inactive_count: 1"]

[John Kennedy]

On Sat, Aug 04, 2018 at 10:56:42PM -0700, Mark Millard via freebsd-arm wrote:
> Rule of thumb figures:
> 564M Active
> vs. RES for the 3 c++'s:
> 119M+191M+226M = 536M for the 3 c++'s.
> 
> So: 564M - 536M = 28M (approx. active for other processes)
> 
> It appears to me that some c++ would likely need to swap out given that
> this context lead to OOM kills.
> 
> (It might be that this rule of thumb is not good enough
> for such judgments.)
> 
> [Personally I normally limit myself to -jN figures that have N*512 MiBytes
> or more on the board. -j4 on a rpi3 or rpi2 has only 4*256 MiBytes.]

  For me, I generally pick a -j# initially based on the number of cores I have
(sysctl -n hw.ncpu), scaling back a bit so the load is somewhere near that
value.  With virtual (hyperthreaded) cores, it'll probably be less.  That
appears to be about right for the RPI3B+ (4 real cores, so -j4, and my load
average is ~4.15 over long periods of time).  Typically less than 50M swapped
out if top is to be believed.

  You'll either be CPU bound or I/O bound, which is what load is crudely trying
to tell you.  You don't want to be wasting time swapping when you could be
writing out finished output.

  I'm not quite masochistic enough to spend all the time it would take to find
out where the point of diminishing returns is for that.

  I think ZFS is doing nice things for me.  I was concerned that it would be
eating too much of my RAM, but I think that compression has been doing nice
things for my I/O performance (particularly the SD card on the RPI3B+).