From nobody Sun Jul 17 07:40:05 2022 X-Original-To: toolchain@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Llxnx6NjWz4WVk4 for ; Sun, 17 Jul 2022 07:40:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Llxnx4bC8z452Z for ; Sun, 17 Jul 2022 07:40:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Llxnx3fNlzrwr for ; Sun, 17 Jul 2022 07:40:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 26H7e56r072014 for ; Sun, 17 Jul 2022 07:40:05 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 26H7e5VI072013 for toolchain@FreeBSD.org; Sun, 17 Jul 2022 07:40:05 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: toolchain@FreeBSD.org Subject: [Bug 265254] lang/gcc11: build gets stuck Date: Sun, 17 Jul 2022 07:40:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mandree@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: toolchain@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback+ X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Maintenance of FreeBSD s integrated toolchain List-Archive: https://lists.freebsd.org/archives/freebsd-toolchain List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-toolchain@freebsd.org X-BeenThere: freebsd-toolchain@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1658043605; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vio+sQdUQVclrbJqk1NTTzprzwjPy3fvc5JvoyRgSuU=; b=ip+4BPLFANf9LG0EP1Tefqn118Ad+TFqPMOaBguG/CQ3Z0fij7C+u47N1OjGOTpAEIXnGR b+Fc0e++O4tUGlgkFk7DwzOe8kqLd2TtSZMF1as8y7qxtC2rLAKm5DwfBWVQPQYu1POyd1 Wt0rIAHeTdKrmUZVVJ87IMCYnHyrwIgRLlQZu92FoNpdDHy0NT04P5iAcU+kQ1zYKLYLBb or3pB0FFAgw+7L60yCa8m0zPD3QSlZMeL0RcRrkH/CGd694gY1NnLmN2emAzykxt0RAuZm 0MGsx+vfnjrb2ZvihZ8X8PYYNhsxy9fqKOlJn9FOxwKkjDphz0cJqHCxHMTu8A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1658043605; a=rsa-sha256; cv=none; b=K8a3xA4SvOE/cXbR6dt1e2ES2QuNo4m7OdEPaPEa7lMbmtmsbQ24gjENKm75ZU4pf/oBzW BphA4v1m+nlycLA5Y8Gf3sk2xrgeTHvywy+9CO+vfmLP+sRLW/S7/cEr81clFlXxdT1rUd bPqRQoQQ/pS7XtbNpc6vW7Il0SWlu1ZaLTXA3Pf/X82tijLCFP/J8yg6M8mo1C7u73IAbG BReprtBw1I+SYgxwECCrOw4vktUrMkZIZTwDlG7bAqCY9ilSQPwqaZ3q3R77D+pYT2WEXL Ie9NpO1mdrdu0TvQlsAy5D/N3CzrgtAOQ17RNdvILZMaGdo3qQZrVH+TNTFNzg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D265254 --- Comment #21 from Matthias Andree --- sorry for the unusual typo quote, I accidentally clicked save changes before proofreading, so here is my same comment written in an intelligible way: ----- So there's some discussion where people are seemingly talking at different levels. Yuri is observing an exploding number of jobs, and if some parent is swapped out under memory pressure - and possibly with ZFS filling up memory and/or disks getting very slow - then the parent process can't collect the children that have exited, hence many zombies.=20=20 Grim process reaper caught up in a traffic jam if you will. Then I have seen some of the GCC drivers (not sure which versions) that att= empt to interface with the GNU make jobservers to avoid that "nested GNU make process explosion", but either this is not configured (in the port, either upstream GCC or FreeBSD's port), or it is not working in a bootstrap =3D> s= omeone could investigate that and I am not sure off-hand if it pertains to GCC11, = or if GCC11 did not have that interface. I recall that GCC used to have a hefty discussion when it was introducing L= TO that it was going slow because there were 16 paperweight compilers that generate intermediate code and then linker ran single-threadedly because at that time (again, not sure which linker exactly) the assumption still was t= hat linking is serial, disregarding the "LTO global optimizer and code generati= on" phases, when LLVM/clang were parallelizing the LTO link, too. I think this= has been fixed in GCC since. Then there are fat and thin LTO variants... So it is a complex matter and GCC11 is not the only offender, apparently mongodb50 was recently told not to use LTO in FreeBSD's port factory settin= g, and there is more.=20 I frequently see my builds killed because my many-GB 8-core 2-threads-per-c= ore Ryzen 1700 fills up the disk during compiler builds, and I FREQUENTLY see multiple compilers competing in poudriere.=20 It's one or two LLVMs, one GCC, and Rust at the same time, and then I usual= ly fill up the disk. Especially if anything generates debugging information intermediately without my setting WITHOUT_DEBUG because someone thought it = wise to have -flto -g or something.=20 Python 3.8 did the latter on my mailserver and could not build with 1 GB RA= M + 1 GB swap, but the Python port has since been fixed. So yes, arguably my own FreeBSD builer VM should have more disk space ;-) But still the enormous LTO resource hunger needs to be addressed in the por= ts tree as a whole. --=20 You are receiving this mail because: You are the assignee for the bug.=