Re: pkg upgrade, vfs.zfs.arc.free_target, vfs.zfs.arc.sys_free, vm.pageout_oom_seq [Notes about error messages during and RAM+SWAP requirements, UFS context]
Date: Tue, 02 Sep 2025 20:49:35 UTC
On Sep 2, 2025, at 11:19, Mark Millard <marklmi@yahoo.com> wrote: > On Aug 31, 2025, at 12:01, Graham Perrin <grahamperrin@gmail.com> wrote: > >> On 31/08/2025 09:38, Mark Millard wrote: >> >>> … >>> >>> So 4 GiBytyes of RAM is insufficient, even with use of: vm.pageout_oom_seq=120 >>> >>> Note: In this context increases is not going to help: it is way too little RAM >>> for pkg. (I was observing via top at the time.) >>> >>> If pkg continues to require such RAM+SWAP usage, FreeBSD is going to end up >>> requiring more than, say, 5 GiBytes of RAM+SWAP, even for UFS contexts. >>> >>>> … >>> >>> For reference, see: >>> >>> https://lists.freebsd.org/archives/freebsd-pkgbase/2025-August/000804.html >>> >>> with details of my test context (where swap was in use). >> >> >> Interesting. >> >> Here, with faster storage for a new guest, 1,024 MB base memory was enough – for both UFS and ZFS – as outlined below. >> >> ---- >> >> >> VirtualBox >> bidirectional shared clipboard >> 1 GB memory >> EFI >> four processors >> PAE/NX >> VBoxSVGA >> FreeBSD-14.3-RELEASE-amd64-dvd1.iso >> disable host I/O cache >> vm.pageout_oom_seq=120 >> UFS >> 3 G swap >> install all components >> pkbasify conversion before first boot (519 packages) >> first boot >> vm.pageout_oom_seq=120 > > The notes here are about the following sequence of 4 commands: > >> pkg install -Fy firefox gitup htop hw-probe kde lsblk lynx nano pciutils plasma6-sddm-kcm roxterm sddm stressdisk uclcmd usbutils virtualbox-ose-additions xfce xorg >> pkg install -Uy firefox gitup htop hw-probe kde lsblk lynx nano pciutils plasma6-sddm-kcm roxterm sddm stressdisk uclcmd usbutils virtualbox-ose-additions xfce xorg >> pkg upgrade -fFqy >> pkg upgrade -fUy > > In a 14.3-Stable amd64 UFS context the above command sequence: > > 1) Will probably OOM on a system with 1 GiByte of RAM but no swap space, > even for the first install command. > > 2) The sequence of 4 commands used (RAM+SWAP): > 1024 MiBytes + 2312 MiBytes (observed maximum) > (So between 3 and 4 GiBytes of RAM+SWAP overall.) > > 3) So will OOM on a system with 2 GiByte of RAM but no swap space, > overall. > > 4) generates the error messages shown by: > > # grep -e ^pkg: -e ^ELF ~/mmjnk-pkg-1GiByteRAM-3p5GiByteSWAPspace-test.txt | sort -u > ELF interpreter /libexec/ld-elf.so.1 not found, error 2 > pkg: Cannot runscript DEINSTALL:No such file or directory > pkg: Cannot runscript POST-DEINSTALL:No such file or directory > pkg: Cannot runscript POST-INSTALL:No such file or directory > pkg: Cannot runscript PRE-INSTALL:No such file or directory > pkg: POST-INSTALL script failed > > # grep -e ^pkg: -e ^ELF ~/mmjnk-pkg-1GiByteRAM-3p5GiByteSWAPspace-test.txt | wc -l > 501 > > All these happen during the "pkg upgrade -fUy". > > I give more details later below. > >> 1353 packages >> 2601 steps >> success >> vm.pageout_oom_seq=12 >> pkg upgrade -fUy >> success >> >> ---- >> >> >> The same with ZFS. Success with, then without, the increased vm.pageout_oom_seq > > > Other information for the text context: > > RAM: 1024 MiByte > RAM+SWAP: 3584 MiByte > vm.pageout_oom_seq: 120 > > # uname -apKU > FreeBSD FBSDamd64S 14.3-STABLE FreeBSD 14.3-STABLE stable/14-n272276-b06973a2f2a8 GENERIC amd64 amd64 1403506 1403506 > > # pkg -v > 2.2.2 > > pkg was the only port-package already installed at the start of > the command sequence. > > > The counts of instances of the 6 individual error message types > were: > > # grep -e ^ELF ~/mmjnk-pkg-1GiByteRAM-3p5GiByteSWAPspace-test.txt | wc -l > 44 > > # grep -e "^pkg:.* DEINSTALL" ~/mmjnk-pkg-1GiByteRAM-3p5GiByteSWAPspace-test.txt | wc -l > 21 > > # grep -e "^pkg:.*POST-DEINSTALL" ~/mmjnk-pkg-1GiByteRAM-3p5GiByteSWAPspace-test.txt | wc -l > 378 > > # grep -e "^pkg: .* POST-INSTALL" ~/mmjnk-pkg-1GiByteRAM-3p5GiByteSWAPspace-test.txt | wc -l > 50 > > # grep -e "^pkg: .* PRE-INSTALL" ~/mmjnk-pkg-1GiByteRAM-3p5GiByteSWAPspace-test.txt | wc -l > 2 > > # grep -e "^pkg: POST-INSTALL script failed" ~/mmjnk-pkg-1GiByteRAM-3p5GiByteSWAPspace-test.txt | wc -l > 6 > > > The RAM+SWAP usage for the sequence progressed as below, > where a top variant monitored system Maximum Observed > Used figures: > > > # pkg install -Fy firefox gitup htop hw-probe kde lsblk lynx nano pciutils plasma6-sddm-kcm roxterm sddm stressdisk uclcmd usbutils virtualbox-ose-additions xfce xorg > . . . > Number of packages to be installed: 834 > . . . > > Swap: 121500Ki MaxObsUsed, > and 1085Mi MaxObs(Active+Wir+Lndry+SwapUsed) > with max 1085Mi (Active+Wir+Lndry+SwapUsed+InAct) at the time > > > # pkg install -Uy firefox gitup htop hw-probe kde lsblk lynx nano pciutils plasma6-sddm-kcm roxterm sddm stressdisk uclcmd usbutils virtualbox-ose-additions xfce xorg > . . . > Number of packages to be installed: 834 > . . . > > Swap: 585012Ki MaxObsUsed, > and 1489Mi MaxObs(Active+Wir+Lndry+SwapUsed) > with max 1521Mi (Active+Wir+Lndry+SwapUsed+InAct) at the time > > > # pkg upgrade -fFqy > > Swap: 1760Mi MaxObsUsed, > and 2726Mi MaxObs(Active+Wir+Lndry+SwapUsed) > with max 2726Mi (Active+Wir+Lndry+SwapUsed+InAct) at the time > > > NOTE: The upgrade command below also involved all the FreeBSD-* base > packages as well. In my context that was all of them: > # pkg info -g "FreeBSD-*" | wc -l > 534 > > # pkg upgrade -fUy > Checking for upgrades (1369 candidates): 100% > Processing candidates (1369 candidates): 100% > Checking integrity... > . . . > Number of packages to be upgraded: 534 > Number of packages to be reinstalled: 835 > [1/2628] Deinstalling FreeBSD-acct-dbg-14.snap20250831004235... > . . . > > Swap: 2312Mi MaxObsUsed, > and 3192Mi MaxObsMaxObs(Active+Wir+Lndry+SwapUsed) > with max 3225Mi (Active+Wir+Lndry+SwapUsed+InAct) at the time > > > So, absent SWAP being enabled, it takes a 4 GiByte system > to avoid OOM activity for the whole sequence in my context. > Otherwise RAM+SWAP needs to be about that big. > > > Other experiments with a different set of port-packages had > an example of needing a 6 GiByte system if SWAP was not > enabled. Otherwise RAM+SWAP needs to be about that big. > > I've no clue what combination of port-packages would > need the most RAM+SWAP for "pkg upgrade -fUy". > > > As stands, various pkg commands may need to have documented > warnings about RAM+SWAP requirements. > Some extra information for excluding FreeBSD-base via "pkg upgrade -r FreeBSD-ports -fUy" . . . Note: the below produced just the 4 error messages: pkg: unlinkat(usr/local/share/mime/x-scheme-handler/): No such file or directory pkg: unlinkat(usr/local/share/mime/uri/): No such file or directory pkg: unlinkat(usr/local/share/mime/print/): No such file or directory pkg: unlinkat(usr/local/share/mime/all/): No such file or directory # pkg upgrade -r FreeBSD-ports -fUy Checking for upgrades (1369 candidates): 100% Processing candidates (1369 candidates): 100% Checking integrity... done (0 conflicting) The following 835 package(s) will be affected (of 0 checked): . . . Number of packages to be reinstalled: 835 [1/1607] Deinstalling firefox-142.0.1_1,2... . . . Swap: 1456Mi MaxObsUsed, and 2314Mi MaxObs(Active+Wir+Lndry+SwapUsed) with max 2416Mi (Active+Wir+Lndry+SwapUsed+InAct) at the time So: max RAM+SwapUsed = 1024 Mi + 1456 Mi == 2480 MiBytes vs. max RAM+SwapUsed = 1024 Mi + 2312 Mi == 3366 MiBytes with FreeBSD-base also involved. === Mark Millard marklmi at yahoo.com