Re: Poudriere insists on using tmpfs for packages listed in TMPFS_BLACKLIST

On Feb 14, 2025, at 12:37, Yusuf Yaman <nxjoseph@protonmail.com> wrote:

> root@hale:~ # pkg info -x poudriere
> poudriere-3.4.2
> root@hale:~ # uname -a
> FreeBSD hale.home.arpa 14.2-RELEASE-p1 FreeBSD 14.2-RELEASE-p1 GENERIC amd64
> 
> -------- Forwarded Message -------- Subject: Poudriere insists on using tmpfs for packages listed in TMPFS_BLACKLIST Date: Fri, 14 Feb 2025 20:07:33 +0300 From: Yusuf Yaman <nxjoseph@protonmail.com> To: FreeBSD Ports ML <freebsd-ports@freebsd.org> 
> 
> Hi,
> 
> I am having a problem where Poudriere (even -devel) does insist on using tmpfs for big packages that i listed in TMPFS_BLACKLIST list in configuration, also TMPFS_BLACKLIST_DIR is set. I am using ZFS. It happens on at least lang/rust and devel/llvm15.

The likes of lang/rust and devel/llvm* use large amounts of file
system space (compared to, say, just 2 GiBytes) it is the larger
areas that are redirected into where you have TMPFS_BLACKLIST_DIR
point, avoiding that also being a tmpfs area. These can be like
17+ GiByte, 25+ GiByte, or more for just one builder in the
TMPFS_BLACKLIST_DIR area. These can be larger than the RAM that
some might have, making having a huge RAM+SWAP be important
absent the TMPFS_BLACKLIST entry, especially if multiple such
builders happen to run in parallel. (There is also a hook for
avoiding any of a list of packages from building in parallel.)

TMPFS_BLACKLIST is not intended to eliminate all tmpfs use by a
a builder, just what most likely potentially grows to be
huge/massive compared to normal: wrkdirs

For reference:

# grep -r TMPFS_BLACKLIST_TMPDIR /usr/local/share/poudriere/
/usr/local/share/poudriere/common.sh: case "${TMPFS_BLACKLIST_TMPDIR:+set}" in
/usr/local/share/poudriere/common.sh: if [ -d "${TMPFS_BLACKLIST_TMPDIR:?}/wrkdirs" ] &&
/usr/local/share/poudriere/common.sh:     ! rm -rf "${TMPFS_BLACKLIST_TMPDIR:?}/wrkdirs/"*; then
/usr/local/share/poudriere/common.sh:     "${TMPFS_BLACKLIST_TMPDIR:?}/wrkdirs"/* || :
/usr/local/share/poudriere/common.sh: rm -rf "${TMPFS_BLACKLIST_TMPDIR:?}/wrkdirs"/* ||
/usr/local/share/poudriere/common.sh: mkdir -p "${TMPFS_BLACKLIST_TMPDIR:?}/wrkdirs"
/usr/local/share/poudriere/common.sh: TMPDIR="${TMPFS_BLACKLIST_TMPDIR:?}/wrkdirs" \

An example of normal/small is ports-mgmt/portmaster ends up using
under 2.5 GiBytes of tmpfs for USE_TMPFS=all .

> 
> Thanks in advance.
> 
> yusuf@hale ~ % mount -v | grep llvm
> yusuf@hale ~ % mount -v | grep tmpfs
> tmpfs on /poudriere/data/.m/142x86-default/ref/.p (tmpfs, local, vnodes: count 36 )
> tmpfs on /poudriere/data/.m/142x86-default/ref/wrkdirs (tmpfs, local, vnodes: count 2 )
> tmpfs on /poudriere/data/.m/142x86-default/ref/var/db/ports (tmpfs, local, vnodes: count 4 )
> tmpfs on /poudriere/data/.m/142x86-default/01/.p (tmpfs, local, vnodes: count 7 )
> tmpfs on /poudriere/data/.m/142x86-default/01/wrkdirs (tmpfs, local, vnodes: count 135024 )
> yusuf@hale ~ %

I'll note that the (unused):

/poudriere/data/.m/142x86-default/01/wrkdirs 

is still present when:

${TMPFS_BLACKLIST_TMPDIR:?}/wrkdirs

is used instead. The:

/poudriere/data/.m/142x86-default/01/wrkdirs

is not deleted and recreated for each builder
to use slot 01 , just avoided for TMPFS_BLACKLIST
usage and emptied before starting a new builder
in the slot.

> Queued: 1 Inspected: 0 Ignored: 0 Built: 0 Failed: 0 Skipped: 0 Fetched: 0 Remaining: 1
>  ID  TOTAL                  ORIGIN   PKGNAME          PHASE TIME     TMPFS     CPU% MEM%
> [01] 00:04:01 devel/llvm15@default | llvm15-15.0.7_10 build 00:03:03 1.55 GiB 99.9%   5%

1.55 GiB indicates that the large amount of file system space was
not placed in TMPFS. The 1.55 GiB appears to have been in/under:
/poudriere/data/.m/142x86-default/01/.p instead.


===
Mark Millard
marklmi at yahoo.com