RE: Is there a way to tell poudriere to allocate more memory to a pkg build?

Dennis Clarke <dclarke_at_blastwave.org> wrote on
Date: Fri, 23 May 2025 17:45:17 UTC :

> I have been watching qt6-webengine-6.8.3 fail over and over and over
> for some days now and it takes with it a pile of other stuff.
> 
> In the log I see this unscripted trash of a message :
> 
> [00:05:03] FAILED: v8_context_snapshot.bin
> [00:05:03] /usr/local/bin/python3.11 
> ../../../../../qtwebengine-everywhere-src-6.8.3/src/3rdparty/chromium/build/gn_run_binary.p
> y ./v8_context_snapshot_generator --output_file=v8_context_snapshot.bin
> [00:05:03]
> [00:05:03]
> [00:05:03] #
> [00:05:03] # Fatal error in , line 0
> [00:05:03] # Oilpan: Out of memory
> [00:05:03] #
> [00:05:03] #

Way to little context so all I can do is basically form
questions at this point.

I assume that you have not explicitly restricted the memory
space for any processes, so that RAM+SWAP is fully available
to everything. If not, you need to report on the details.

How much RAM? How much SWAP space? (So: how much RAM+SWAP?)
(RAM+SWAP does not vary per process tree or per builder,
presuming no deliberate restrictions have been placed.)

Do you even have "whatever it seems to want" configured
for the RAM+SWAP? (I'm guessing that you do not know that
the "128G" figure is in fact involved.)

How many parallel builders are active in the bulk run
as the bulk build approaches the failure?

How much RAM+SWAP in use by other builders or other things
on the system as the system progresses to that failure (not
after the failure)?

What USE_TMPFS= setting is in use? What about TMPFS_BLACKLIST
use? (So: is v8_context_snapshot.bin all being put in
RAM+SWAP? Put on media instead?)

I'll note that even inactive builders can hold tmpfs
space that had been in use in the prior build in the
builder. So looking at such use could potentially be
relevant.

Any general other system use of tmpfs that might be competing
for RAM+SWAP to some significant degree?

ZFS (and its ARC)? UFS? If ZFS: Any tuning?

Basically: all the significant sources of competing for
RAM+SWAP?

Watching top as the system approaches the failure and
reporting on what it showsd could be useful.

Going in another direction: notes appropriate for other
folks to try to well approximate the the context and then
try to replicate the issue?

> I can not make that stuff up.
> 
> So is there a way to say that the pkg build for a particular thing
> is allowed to use 128G of memory or whatever it seems to want?

It goes the other way: you can prevent other things for
competing for RAM+SWAP. Also, you may be able to increase
RAM+SWAP (but avoiding mistuning via too much SWAP for
the amount of RAM). (For 64-bit contexts, the system
tends to complain somewhat above, say, SWAP=3.6*RAM,
so above RAM+SWAP=4.6*RAM . The detailed figure varies
some from build to build and the "3.6" has some margin
to avoid detailed tracking of the variations.)



===
Mark Millard
marklmi at yahoo.com