Re: swap_pager: cannot allocate bio

(Sorry that the subject on this thread may not be relevant any more, but I don’t want to disconnect the thread.)

> On Nov 15, 2021, at 13:17, Chris Ross <cross+freebsd@distal.com> wrote:
>> On Nov 15, 2021, at 10:08, Andriy Gapon <avg@freebsd.org> wrote:
> 
>> Yes, I propose to remove the wait for ARC evictions from arc_lowmem().
>> 
>> Another thing that may help a bit is having a greater "slack" between a threshold where the page daemon starts paging out and a threshold where memory allocations start to wait (via vm_wait_domain).
>> 
>> Also, I think that for a long time we had a problem (but not sure if it's still present) where allocations succeeded without waiting until the free memory went below certain threshold M, but once a thread started waiting in vm_wait it would not be woken up until the free memory went above another threshold N.  And the problem was that N >> M.  In other words, a lot of memory had to be freed (and not grabbed by other threads) before the waiting thread would be woken up.
> 
> Thank you both for your inputs.  Let me know if you’d like me to try anything, and I’ll kick (reboot) the system and can build a new kernel when you’d like.  I did get another procstat -kka out of it this morning, and the system has since gone less responsive, but I assume that new procstat won’t show anything last night’s didn’t.

I’m still having this issue.  I rebooted the machine, fsck’d the disks, and got it running again.  Again, it ran for ~50 hours before getting stuck.  I got another procstat-kka off of it, let me know if you’d like a copy of it.  But, it looks like the active processes are all in arc_wait_for_eviction.  A pagedaemon is in a arc_wait_for_eviction under a arc_lowmem, but the python processes that were doing the real work don’t have arc_lowmem in their stacks, just the arc_wait_for_eviction.

Please let me know if there’s anything I can do to assist in finding a remedy for this.  Thank you.

             - Chris