Re: poudriere bulk -a fails on UFS: "Too many links" under logs/bulk/latest-per-pkg/ and then "Failed: starting"

On Nov 4, 2023, at 13:56, Tatsuki Makino <tatsuki_makino@hotmail.com> wrote:

> Hello.
> (I am replying to freebsd-ports only.)

Seems reasonable.

> Mark Millard wrote on 2023/11/04 21:00:
>> # ls -a /usr/local/poudriere/data/logs/bulk/latest-per-pkg/ | wc -l
>>   32767
> 
> The number of packages will be inflated relative to the number of origins because FLAVORS uses PKGNAMEPREFIX and PKGNAMESUFFIX.

/usr/local/poudriere/data/logs/bulk/latest-per-pkg/ contains direcotries
that have the names of the packages with log files form bulk runs, not
origins.

> Therefore, something like BUILD_ALL_PYTHON_FLAVORS should not be used continuously for purposes other than development and debugging.

The from scratch bulk -a run is a kind of rare experiment that I do, not
a regular thing. It is deliberately a simple bulk -a run when I start
such an experimental sequence.

> Ports for which the origin directory no longer exists, and whose description no longer exists from MOVED (e.g. www/seamonkey), seem to remain in the package forever.
> It is better to run poudriere-pkgclean regularly.

The sequence used to start my experiments is as shown below:

#! /bin/sh
poudriere pkgclean -jmain-amd64-bulk_a -A
poudriere bulk -jmain-amd64-bulk_a -a

So pkgclean had been run. However, as I remember, this does
not manage log files. I do that separately.

For /usr/local/poudriere/data/logs/bulk/latest-per-pkg/
being empty at the start, the failure would still happen.

The 32767 is a UFS limitation via a 16 bit field associated with:

/usr/include/ufs/ufs/dinode.h:#define UFS_LINK_MAX 32767

Specifically di_nlink in sys/ufs/ufs/dinode.h :

struct ufs2_dinode {
	uint16_t	di_mode;	/*   0: IFMT, permissions; see below. */
	int16_t		di_nlink;	/*   2: File link count. */
. . .

Having more than 32767 packages attempted to be built
leads to:

/usr/local/poudriere/data/logs/bulk/latest-per-pkg/

trying to go beyond that UFS_LINK_MAX limit in what latest-per-pkg/
contains.

ZFS does not have the limitation and the prior from-scratch ZFS
based bulk -a experiment was able to finish its build. bulk -a
in the ZFS context historically used somewhat more RAM+SWAP space
than a UFS based bulk -a would.

> Would checking that area reduce that number?

For amd64, there are over 32767 packages that should get directories
in /usr/local/poudriere/data/logs/bulk/latest-per-pkg/ . The usage
poudriere puts them to counts them in di_nlink in UFS's directory inode
and the UFS limitation is reached before all the packages are present.
/usr/local/poudriere/data/logs/bulk/latest-per-pkg/ ends up just not
fitting in UFS's limitations for how things are currently organized.

I used to be able to compare/contrast the ZFS vs. UFS RAM+SWAP usage
for bulk -a runs. But last I did such was before the builds going
beyond 32767 packages with logs.

(I've not checked if /usr/local/poudriere/data/logs/bulk/latest-per-pkg/
is the only issue. It is just the first context to stop the build.)

===
Mark Millard
marklmi at yahoo.com