RPI3 swap experiments ["was killed: out of swap space" with: "v_free_count: 5439, v_inactive_count: 1"]

[bob prohaska]

On Mon, Aug 06, 2018 at 11:58:37AM -0400, Mark Johnston wrote:
> 
> If the above suggestion doesn't help, the next thing to try would be to
> revert the oom_seq value to the default, apply this patch, and see if
> the problem continues to occur.  If this doesn't help, please try
> applying both measures, i.e., set oom_seq to 120 _and_ apply the patch.
> 

Applying both measures resulted in a panic, not entirely unlike the effect
of deliberately running the machine out of swap on microSD. The first obvious
sign of trouble was errors attributed to da0:

(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 00 25 bc b8 00 00 80 00 
(da0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
(da0:umass-sim0:0:0:0): Retrying command, 3 more tries remain

but instead of going on endlessly a panic followed a few lines later:

swap_pager: I/O error - pageout failed; blkno 47015,size 16384, error 5
ending with 
panic: vm_thread_swapin: cannot get kstack for proc 615

It appears that the gstat script stopped before this, since these console
messages never made it to swapscript.log.

What I could think of to capture is at
http://www.zefox.net/~fbsd/rpi3/swaptests/r337226M/1gbsdflash_1gbusbflash/batchqueue/pageout120/

The machine accepted a reboot command at the serial console and came up without
any manual intervention.

Thanks for reading, any suggestions on what to try next are welcome!

bob prohaska