Re: Can not build kernel on 1GB VM
- Reply: Michael Wayne : "Re: Can not build kernel on 1GB VM"
- In reply to: Michael Wayne : "Re: Can not build kernel on 1GB VM"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Mon, 16 May 2022 19:07:18 UTC
On 2022-May-16, at 07:37, Michael Wayne <firstname.lastname@example.org> wrote: > More info. I am running UFS so the ZFS should not be an issue > > % pstat -s > Device 1K-blocks Used Avail Capacity > /dev/md99 1048576 0 1048576 0% That may well explain some (or all?) of what is going on: file-system/vnode backed swap spaces are subject to deadlocks. On 2017-Feb-13, at 7:20 PM, Konstantin Belousov <kostikbel at gmail.com> wrote on the freebsd-arm list: QUOTE . . . swapfile write requires the write request to come through the filesystem write path, which might require the filesystem to allocate more memory and read some data. E.g. it is known that any ZFS write request allocates memory, and that write request on large UFS file might require allocating and reading an indirect block buffer to find the block number of the written block, if the indirect block was not yet read. As result, swapfile swapping is more prone to the trivial and unavoidable deadlocks where the pagedaemon thread, which produces free memory, needs more free memory to make a progress. Swap write on the raw partition over simple partitioning scheme directly over HBA are usually safe, while e.g. zfs over geli over umass is the worst construction. END QUOTE I only use swap partitions for swap space --in order to avoid the deadlocks in parts of the kernel operation that I ran into otherwise. > % df > Filesystem 1K-blocks Used Avail Capacity Mounted on > /dev/ufs/rootfs 25388500 17630840 5726580 75% / > devfs 1 1 0 100% /dev > > On Sat, May 14, 2022 at 08:02:29AM -0700, Mark Millard wrote: >> The way to know if out of swap might actually be involved >> in the context are some other messages that do not of >> themselves announce kills: >> >> swap_pager: out of swap space >> swp_pager_getswapspace(. . .): failed >> >> If you are getting either of those 2, then you are actually >> running out of swap space. Otherwise you are not. > > I am not getting either of those. Watching swap as ctfmerge runs up > confirms that I am likely not running out of swap. > > Note also that I continue to run a kernel built with the suggested patch to > /sys/vm/vm_pageout.c (The oldest code I cover for extra messaging relative to OOM kills and and the like is releng/13.0 code these days. As I remember, the code has been moved around between files some since 12.x . So the below may read odd for your context.) Which suggested patch(s)? Any patches for . . . sys/vm/swap_pager.c ? swp_pager_meta_build: swap blk uma zone exhausted swp_pager_meta_build: swap pctrie uma zone exhausted sys/vm/vm_fault.c ? vm_fault_allocate_oom: proc %d (%s) failed to alloc page on fault, starting OOM sys/vm/vm_page.c ? thread %d waiting for memory For sys/vm/vm_pageout.c I have messages: vm_pageout_mightbe_oom: kill context: v_free_count: %u, v_inactive_count: %u, v_laundry_count: %u, v_active_count: %u (The 2 forming one line of output.) failed to reclaim memory a thread waited too long to allocate a page (The 2 above are now the standard messages in releng/13.1 , stable/13 , and main .) swblk or swpctrie zone exhausted (In releng/13.1 , stable/13 , and main this is still shown as the mismoner "out of swap space".) >> If you are not the real reason is one of 4: > > Likely the swap device isn't responding within a "reasonable" time. (from another mail in this chain) > >> FYI: >> >> kernel: swap_pager: indefinite wait buffer: bufobj: . . ., blkno: . . ., size: . . . >> >> is for a swap read taking over 20 seconds (including >> time when queued but waiting in the queue to start >> the transfer). > > I got this message on 12.1. I am not getting it on 12.3 > > >> /boot/loader.conf can use the likes of: >> >> vm.pageout_oom_seq=120 >> vm.pfault_oom_attempts=-1 >> #vm.pfault_oom_attempts= 3 >> #vm.pfault_oom_wait= 10 >> >> I do not know if you have tried any of these. > > Quoting one of my old emails: > On 12.1 I used sysctl to set >> vm.pageout_oom_seq=120 >> vm.pfault_oom_attempts=-1 >> >> There was no improvement. I still see processes getting killed due >> to no swap space despite only 7-8 MB being reported used. It sorta >> feels like it's not really able to use swap at all. >> >> Note that everything worked fine on 11.x, this is a new issue on 12. > > I have not tried this on 12.3, nor have I tried the other two. Will do. > Can you switch to using a swap partition instead of file-system/vnode backed swap space? === Mark Millard marklmi at yahoo.com