How poudriere's PACKAGE_FETCH_WHITELIST should work?

From: Miroslav Lachman <000.fbsd_at_quip.cz>
Date: Wed, 15 Feb 2023 19:50:59 UTC
Poudriere and fetching build dependecies - I would like to know how it 
is supposed to work?

I use poudriere a bunch of years, with ports overlays etc. As I need to 
rebuild packages for our machines many times a month I am tired of 
building huge and slow packages like llvm, gcc, rust (as they often eat 
all memory+swap then build is killed by OOM).

I tied to setup PACKAGE_FETCH with the two following variables in 
poudriere.conf
   PACKAGE_FETCH_BRANCH=quarterly
   PACKAGE_FETCH_WHITELIST="gcc* rust llvm* gcc90 gcc10 gcc11 gcc12 
gcc13 gcc14 llvm10 llvm11 llvm12 llvm13 llvm14 llvm15 lua54"

When I started usual "poudriere bulk" only gcc12 were fetched, llvm10 
and rust built from sources:

   [00:01:02] Package fetch: Will fetch 1 packages from remote or local 
pkg cache
   The following packages will be fetched:
   New packages to be FETCHED:
         gcc12: 12.2.0_5 (81 MiB: 100.00% of the 81 MiB to download)
   Number of packages to be fetched: 1

I thought maybe I have some different options selected for llvm and rust 
than the default on official FreeBSD packages, I double checked, removed 
stored options and started another poudriere bulk with different package 
set (llvm10 and rust will be needed for the set).

This time the rust package was downloaded, but llvm10 built from source 
again:

   [00:00:22] Package fetch: Will fetch 1 packages from remote or local 
pkg cache
   The following packages will be fetched:
   New packages to be FETCHED:
         rust: 1.66.0 (112 MiB: 100.00% of the 112 MiB to download)
   Number of packages to be fetched: 1
   The process will require 112 MiB more space.
   112 MiB to be downloaded.
   [xxxxx] Fetching rust-1.66.0.pkg: 100%  112 MiB  39.2MB/s    00:03

But the mystery is that "poudriere bulk" failed on building rust even if 
it should be used from fetched package:

   [04:11:44] Failed ports: lang/rust:build
   [04:11:44] Skipped ports: devel/cargo-c graphics/libimagequant 
graphics/py-pillow@py37

I checked the logs and the rust build process was killed by OOM killer. 
(but why poudriere was building it if it was already fetched?)

I started poudriere bulk again, rust was fetch again a this time it was 
really used to build other packages, no rebuild of rust from sources needed.


I am really confused why Poudriere fetches only 1 package at a time if 
it should fetch gcc12, rust and llvm10?

Why Poudriere tried to build rust if it fetches it as pkg?

And why it does not fetch llvm10 even if it is available and we do not 
have options stored for devel_llvm10?

Kind regards
Miroslav Lachman