[Bug 265254] lang/gcc11: build gets stuck

From: <bugzilla-noreply_at_freebsd.org>
Date: Sun, 17 Jul 2022 07:40:05 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265254

--- Comment #21 from Matthias Andree <mandree@FreeBSD.org> ---
sorry for the unusual typo quote, I accidentally clicked save changes before
proofreading, so here is my same comment written in an intelligible way:
-----
So there's some discussion where people are seemingly talking at different
levels.

Yuri is observing an exploding number of jobs, and if some parent is swapped
out under memory pressure - and possibly with ZFS filling up memory and/or
disks getting very slow - then the parent process can't collect the children
that have exited, hence many <defunct> zombies.  

Grim process reaper caught up in a traffic jam if you will.

Then I have seen some of the GCC drivers (not sure which versions) that attempt
to interface with the GNU make jobservers to avoid that "nested GNU make
process explosion", but either this is not configured (in the port, either
upstream GCC or FreeBSD's port), or it is not working in a bootstrap => someone
could investigate that and I am not sure off-hand if it pertains to GCC11, or
if GCC11 did not have that interface.

I recall that GCC used to have a hefty discussion when it was introducing LTO
that it was going slow because there were 16 paperweight compilers that
generate intermediate code and then linker ran single-threadedly because at
that time (again, not sure which linker exactly) the assumption still was that
linking is serial, disregarding the "LTO global optimizer and code generation"
phases, when LLVM/clang were parallelizing the LTO link, too.  I think this has
been fixed in GCC since.

Then there are fat and thin LTO variants...

So it is a complex matter and GCC11 is not the only offender, apparently
mongodb50 was recently told not to use LTO in FreeBSD's port factory setting,
and there is more. 



I frequently see my builds killed because my many-GB 8-core 2-threads-per-core
Ryzen 1700 fills up the disk during compiler builds, and I FREQUENTLY see
multiple compilers competing in poudriere. 

It's one or two LLVMs, one GCC, and Rust at the same time, and then I usually
fill up the disk. Especially if anything generates debugging information
intermediately without my setting WITHOUT_DEBUG because someone thought it wise
to have -flto -g or something. 

Python 3.8 did the latter on my mailserver and could not build with 1 GB RAM +
1 GB swap, but the Python port has since been fixed.

So yes, arguably my own FreeBSD builer VM should have more disk space ;-)


But still the enormous LTO resource hunger needs to be addressed in the ports
tree as a whole.

-- 
You are receiving this mail because:
You are the assignee for the bug.