Re: What can I learn about data that is staying paged out? (There is a more specific poudriere bulk related context given.)
Date: Sat, 11 Jun 2022 22:42:24 UTC
I got evidence finally. An example is the sequence: # swapoff /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16 # swapon /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16 (Wait for a time letting some "staying paged out" space accumulate.) # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ 319 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ (Wait for a time letting some more "staying paged out" space accumulate.) # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ 307 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ (The size decreases seem to roughly track the sustained Swap Used increases. [There is some other paging activity but it is not a lot.]) # swapoff /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16 # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ 379 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ (Apparently, du -xsm does not count the paged out pages for tmpfs but the pages show up when forced back into RAM.) # swapon /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16 # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ 379 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ (I expect that when I see the swap usage grow in a staying paged out manor, I'll find that this -xsm figure will also have decreased on the retry in the new context.) The primary space in question is actually in a subdirectory: Taking a quick estimate of the file count in . . ./ref/.p/var/cache/ I get: # ls -Tla /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | wc 82146 821452 7942485 So something like 82144 files ( ignoring . and .. ). # ls -Tla /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | more total 301234 drwxr-xr-x 2 root wheel 5257152 May 28 02:31:26 2022 . drwxr-xr-x 4 root wheel 128 May 28 01:56:42 2022 .. -rw-r--r-- 1 root wheel 2 May 28 02:26:40 2022 builder_active%01 -rw-r--r-- 1 root wheel 2 May 28 02:31:24 2022 builder_active%02 -rw-r--r-- 1 root wheel 2 May 28 02:31:26 2022 builder_active%03 -rw-r--r-- 1 root wheel 2 May 28 02:31:26 2022 builder_active%04 -rw-r--r-- 1 root wheel 18 May 28 01:56:51 2022 originspec-pkgname%accessibility_accerciser -rw-r--r-- 1 root wheel 19 May 28 01:56:51 2022 originspec-pkgname%accessibility_at-spi2-atk -rw-r--r-- 1 root wheel 20 May 28 01:56:51 2022 originspec-pkgname%accessibility_at-spi2-core -rw-r--r-- 1 root wheel 11 May 28 01:56:51 2022 originspec-pkgname%accessibility_atk -rw-r--r-- 1 root wheel 13 May 28 01:56:51 2022 originspec-pkgname%accessibility_atkmm -rw-r--r-- 1 root wheel 17 May 28 01:56:51 2022 originspec-pkgname%accessibility_caribou . . . -rw-r--r-- 1 root wheel 17 May 28 01:56:55 2022 pkgname-originspec%zutils-1.11 -rw-r--r-- 1 root wheel 15 May 28 02:03:32 2022 pkgname-originspec%zxfer-1.1.7 -rw-r--r-- 1 root wheel 14 May 28 02:03:00 2022 pkgname-originspec%zxid-1.42_1 -rw-r--r-- 1 root wheel 19 May 28 02:04:03 2022 pkgname-originspec%zxing-cpp-1.3.0 -rw-r--r-- 1 root wheel 12 May 28 01:59:41 2022 pkgname-originspec%zydis-3.1.0 -rw-r--r-- 1 root wheel 18 May 28 01:57:13 2022 pkgname-originspec%zynaddsubfx-3.0.6,2 -rw-r--r-- 1 root wheel 9 May 28 02:02:07 2022 pkgname-originspec%zyre-2.0.1 -rw-r--r-- 1 root wheel 14 May 28 01:59:41 2022 pkgname-originspec%zziplib-0.13.72_1 -rw-r--r-- 1 root wheel 14 May 28 02:03:00 2022 pkgname-originspec%zzuf-0.13_1 -rw-r--r-- 1 root wheel 13 May 28 01:56:45 2022 ports_metadata%top_git_hash -rw-r--r-- 1 root wheel 4 May 28 01:56:49 2022 ports_metadata%top_unclean Each file seems to have a 4096 Byte page used, making for something around 320 MiBytes. Much of it not being heavily used, some of it is gradually paged out to make room for other things. # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ 326 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ (Still no reported swap usage.) For reference: # du -xsAm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ 46 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ === Mark Millard marklmi at yahoo.com