From nobody Wed Feb 22 02:32:42 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PM0dq0VJWz3s6LT for ; Wed, 22 Feb 2023 02:36:15 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PM0dn6RQTz3hnZ for ; Wed, 22 Feb 2023 02:36:13 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org designates 2a0b:f840::12 as permitted sender) smtp.mailfrom=pmc@citylink.dinoex.sub.org; dmarc=none Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840:0:0:0:0:0:12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 31M2a6nI031510 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 22 Feb 2023 03:36:06 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: (from uucp@localhost) by uucp.dinoex.org (8.17.1/8.17.1/Submit) with UUCP id 31M2a6on031509; Wed, 22 Feb 2023 03:36:06 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from disp.intra.daemon.contact (disp-e.intra.daemon.contact [IPv6:fd00:0:0:0:0:0:0:112]) by admn.intra.daemon.contact (8.17.1/8.17.1) with ESMTPS id 31M2Xihk043529 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=OK); Wed, 22 Feb 2023 03:33:45 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from disp.intra.daemon.contact (localhost [127.0.0.1]) by disp.intra.daemon.contact (8.17.1/8.17.1) with ESMTPS id 31M2WgeU003962 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 22 Feb 2023 03:32:42 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: (from pmc@localhost) by disp.intra.daemon.contact (8.17.1/8.17.1/Submit) id 31M2WgYb003961; Wed, 22 Feb 2023 03:32:42 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-Authentication-Warning: disp.intra.daemon.contact: pmc set sender to pmc@citylink.dinoex.sub.org using -f Date: Wed, 22 Feb 2023 03:32:42 +0100 From: Peter To: freebsd-stable@freebsd.org Cc: Mark Millard Subject: [analysis] Re: 13.2 BETA2: how do debug META_MODE? Message-ID: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Milter: Spamilter (Reciever: uucp.dinoex.org; Sender-ip: 0:0:2a0b:f840::; Sender-helo: uucp.dinoex.org;) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [IPv6:2a0b:f840:0:0:0:0:0:12]); Wed, 22 Feb 2023 03:36:08 +0100 (CET) X-Spamd-Result: default: False [-2.29 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.991]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sub.org]; FREEMAIL_CC(0.00)[yahoo.com]; RCVD_COUNT_FIVE(0.00)[5]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_XAW(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[] X-Rspamd-Queue-Id: 4PM0dn6RQTz3hnZ X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N It appears as my issue goes away by itself at the fourth subsequent build. ># cd /usr/src/ ># env WITH_META_MODE=yes make buildworld ># env WITH_META_MODE=yes make installworld ># env WITH_META_MODE=yes make buildworld (again #0) >## no more rebuilds below? ># env WITH_META_MODE=yes make buildworld (again #1) ># env WITH_META_MODE=yes make buildworld (again #2) > >"again #0" will rebuild llvm/clang. The other two "again"s >will not. Strangely, I observed rather the opposite: the issue was with those nodes that do *not* get installed. Those that do get installed, they behaved as expected. Details: When starting a build, there are some programs in /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ from the last build. These might be suitable for the new build (or they might not, who knows) - but then, very early in the buildworld, many of these files are copied from the running base system, with their mtime preserved. So now these files do not have an mtime of the last build, but of the last *install* of this running instance. (And that may be too late.) This is done here - and I have no idea how these files are selected: > -------------------------------------------------------------- > >>> Rebuilding the temporary build tree > -------------------------------------------------------------- [...] > Building /usr/obj/usr/src/amd64.amd64/tools/build/host-symlinks > Linking host tools into /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin Over all, /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ tends to become a zoo: some files are copied from the base system, some are installed during the build, some are not replaced at all. I tried to figure what exactly is happening, with the example of the make-roken + roken.h. I start the upgrade to 13.2 with an empty obj tree (because after switching and rebasing branches, my mtimes are recreated from the latest commit - and they will certainly not align with the old ones). In the first build the binaries in the obj tree have to be created. But since the running system is still 13.1, these binaries get all stamped for 13.1. In the second build -now running 13.2- META_MODE decides that the mtimes are now fine and does not rebuild. META_MODE has no notion of the currently running OS version, it only considers mtimes and cannot detect that these dependencies are stamped with an old OS version. In the third build META_MODE finds some mtimes obsolete again, and does rebuild toolchain binaries. This would not have any effect because "install" would not copy them to the bin-dir if the same binaries are already there. But in this scenario the old binaries are from 13.1, so they are different, and "install" copies the new ones in - and now everything that somehow depends on them will also rebuild. Finally in the fourth build everything appears to be fine. Conclusion: For a safe upgrade (specifically for a major version change) it is not so much necessary to delete the obj tree before the first build, but rather *after* the first build, i.e. after the base system has been upgraded. Some timings: The base gets installed into a clean DESTDIR and used for the next pass. The obj-trees are individually kept. Initial (obj trees deleted) ( 4 vcore) 230217231308.base.pass1.sst: real 9339.64 base w/ kernels 230217231308.base.pass2.sst: real 7982.69 230217231308.admn.pass1.jail.sst:real 9346.90 jail w/ compiler 230217231308.admn.pass2.jail.sst:real 5460.16 230217231308.data.pass1.jail.sst:real 4094.04 jail w/o compiler 230217231308.data.pass2.jail.sst:real 143.39 230217231308.iamk.pass1.jail.sst:real 8050.27 jail w/ compiler 230217231308.iamk.pass2.jail.sst:real 5226.32 230217231308.oper.pass1.jail.sst:real 2910.28 jail w/o compiler 230217231308.oper.pass2.jail.sst:real 92.05 230217231308.rail.pass1.jail.sst:real 3236.29 jail w/o compiler 230217231308.rail.pass2.jail.sst:real 99.49 230217231308.tele.pass1.jail.sst:real 3170.34 jail w/o compiler 230217231308.tele.pass2.jail.sst:real 180.65 pass3 (10 vcore) 230222000242.base.std.sst: real 1162.80 base w/ kernels 230222000242.admn.std.jail.sst:real 1759.15 jail w/ compiler 230222000242.data.std.jail.sst:real 155.54 jail w/o compiler 230222000242.iamk.std.jail.sst:real 1715.07 jail w/ compiler 230222000242.oper.std.jail.sst:real 149.51 jail w/o compiler 230222000242.rail.std.jail.sst:real 151.73 jail w/o compiler 230222000242.tele.std.jail.sst:real 150.52 jail w/o compiler pass4 (10 vcore) 230222021535.edge.std.sst: real 1018.79 base w/ kernels 230222021535.admn.std.jail.sst:real 101.61 jail w/ compiler 230222021535.data.std.jail.sst:real 67.47 jail w/o compiler 230222021535.iamk.std.jail.sst:real 100.91 jail w/ compiler 230222021535.oper.std.jail.sst:real 66.52 jail w/o compiler 230222021535.rail.std.jail.sst:real 68.00 jail w/o compiler 230222021535.tele.std.jail.sst:real 66.54 jail w/o compiler