From nobody Wed Feb 22 03:31:01 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PM1sP1H2Dz3s99s for ; Wed, 22 Feb 2023 03:31:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-23.consmr.mail.gq1.yahoo.com (sonic303-23.consmr.mail.gq1.yahoo.com [98.137.64.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PM1sN5tBPz3pbR for ; Wed, 22 Feb 2023 03:31:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677036678; bh=xkyWksJABAoUrKD23tnyPHZCAU/Eq2pjWFmokROrVa8=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=k/4VuUw4xguJA+YjWdo7koZB3yBffxdK0AcXLTyfkY8w8De+CzeHnHqqencsKZy3X86rvBhCV/llIbGtB2CXhjenR2SZOiNUqnI069smZOzfZ2sP/oqdsKXzK07bD6DbHe1u7rzLpIm3f3C0oMH81D+tx94TZyqhGHDqmVIVvhVvmqKzNw5n+ww0msazJQ5SHL6B+x3t8tJPSCixwgveufI2KHSreyW0Ji2nw01f8U0dYas1DS3RN1sapSxEA3FTA1xeJn9whOPgK2ABNWMKOKud2Fmr1F+F6zkABkffd5qoxRZ4rE8e+CLavgL/TyAoEuRneyOGN3WZYhjVrUqH0Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677036678; bh=X5g4DMDbZoFTBpZ8ltN+KhuVJF7yAqbHiWlhNmXYv2z=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=JnIVWZB26WRwA9DjNHtHa0+k13kx3rU5qH3EqytALnqkeywWLO6f5ZHKX6aBS65Gdzv1uX0tSre2J3xykSPLq870Dq4/T3H+Zn5cSmmRzBF5VBttLrHWojF+4dfe4NiOERsX75kjvxMYXGUt5/4ckE7v/H0sXEUfaKZYyW+IR1TrcYIV+AQvDDhYUgsK68z8FGkf5RSSNvcYDmhsIbPMQ5OKwZqjCGKCt9vUGUWHkn5cGnZQlzWbdJL2+97JIiHO+tB4JjLypXufupdnCIy+r2DfDPnwzeSpDQpd3K+Hvu14uQ9q7KDv3Egv3I2pMn1uAh9d8W6NCh5bTFTZ+R4p6A== X-YMail-OSG: Ucbeug8VM1kOw4zOeOOpmgr8DujWn8YBdjLEROeunmBRKJPHBmy_Y8QFcC6IdzI foBiwJdo9ix9S6Vczz9lO4IvQOz2x0ymIsmS.gnrJcF.DIZswcbwBkhyCDlPxVizNYEkDwkgrvSv vtl2P.c5KZSaNGLJNo9X2LbU2kiq022kwuBO3VH3oV53lfw.0Ik19ZYDGaiuM9HCaHQ9c3bCNTOd 471lA1fsi84lFYz5Yf9IewzH2DrV7A9freLkS1f37Qncs0ABDpwzD55zUU2YbXV9Vp139tEx5NVz 1jjX0CK_DxHGINmZ1.aCUI8sdKAgbzIJnE4mBB_K2evD3FuWwFHpDwPxEsdp0ATSB0ASjYGWf7bB LVZGxc72DlfLmYiDuBnJhmGvAmlmHdVennsZCD3kjEqTd1CWe2aYW9Ffr9.oDYgStCSrcasPM8jW FKJPfZigPTjFLbFezCbuAGPiDRQcJIKIDMNK2tBYMoyz7641ssa0MUSxIoD6TlOqntHQ5PcFYqpB n3wRHzhcE3dSTzHGfyWIVycFeZUcV9IVDv3gCAcZ.mk8Bw26psiXAVEajyxhra2fhwuNfwGx4Grm kDtkl3HXaiupo6PCmDf4n8VVGXOvkFWLF1tujRYXWmfP032FlxrR8anACgzFjK8DLptVYqZPzN1M yku87jig6nwtykbXea0zEIaMOKvt._9llXFSfZbPIksehqTOAeBPLb6pucW8dxkApr7t3BJGGWxa T2alG5fs9PMk9otVx0V3t13P2NbX9HvoYWPb0dW6KH6vmxD1a_zoutZwKSt5K.f4PqwVOVw4F.su EajHWvKzKvQeDLPEZZcLz6T8R3DISQBAkF43E0pbPnxnptWCvDyr_BiZu5dowI04n6Y_qRNAeNV2 .x6xm.rHiqo5QCT5IR_AZZQm31w8QGGO2.DBfx7batywEvz4XRf5GJrkgmW.5XflauZhxF6ezHpo YACKtiLysGWd8Fc87FjDGS7qUUZICihsJc4oKjCnbLui_qI52NvqRNDqn_RHanx8kKv.A1ApnrMj pZeN1IoSSSBEE8LmqCqG5mEX98UEsY_rKEFu8RuX.8JJwKEgB.RTYEZWDocRQwuICw1D2umni_oT 7aKk58LTq0EHTIZALFaOLVzL2Jw66OKM8l1BmXkxrAWE0JXV_Bz8UMUe_61WqEyJrqxU3610xyFC lOorYucdVqKTmz1y3B.Oc0jgiu8YD3e2wWqqhgk6zi23iHfZHv1WnGsXx3kGYNDFQfjssOuZLouB wEyo4_.E7eQu4FLw2605JAj4hZqQfN4YCexspQThWj0CjHRchAsnSr4K_XoNw6Li88IMOx1jWfHZ gdqf8tsi4KRzvAR0PYv2QBnZ9ijQZhmITfAVa2FX0klhXGLSvEin4LqLHGhwFr984FeyP5Yz0w1C DoHGB.8Vce.A9txsGbdM7OFVFRe4QkxrMOnPE_rerhlsCbrUKzAYHN99.dEDyxmZKguYC7isHtYX qw5TCAzeTvreG9gpZkXmaN0bQKm25q6sFS8Tqy9F1g0gnBLnuQJ7o5QsFhc2ObwzFI8fBosEzANu erFV1jD4px6BZBvxEU5utrmnfm2haXyNGaNjApQZh6TLUIo9TaQtNriuoQTOAxVOT5s4Vml3sNyL L7lRAZjHq4g7Zeqv72XMbl0SPNdavNvtJbpgMvSo44JVdqNUTFP5lZYlQyhI_8OCweE4NVzVeGlh yneo.PtK5V6rYTjnPRGlRJtEtNt.5bdh7BvyIt0xpZgGIPM1rdz88WoLkH6I4M9TfYp1fKYR3OsC UkEwg6XEMZb5tvXjGOhoYh.M5GqjEFdxXPbhVLFWmso.vzkxcZg_66ofnRAyC7gj63oQC16CZ.TP fJcI0yXeaLKW7NMjwsb8Msip3pcDHzKyWPt2OB1X1VV7gdt7VQIfjvscqfTvsohl6hne1Hhk1GUt Jsk_nuUlUc2nEW9MhBIdgxKWGJsGgc05Ym.Jv.o9EQhXoCMFhVsJAsaU9t6yXhRjWPgT8lb2GTF_ FPjCz8CYx16tJU3Yfe3zNOFHAkpf.okqu_3cKmI5j.IYs_eODoRg8prvAbEx1eUnfhcmFE20gcP_ dxT97bo086XabEinJID32zSDI6cc9VmDq6FVvD7cTBQJYkHh58b2Nd9DYJexv1UBxp6I.le6qMG8 umt2sYrENWZ6WE9QSYt9tRyg3..IFkcnskLwiYUNblIPnJXv5nwDZk0iIwbsH8JiueU9C6xJK_iG ucfmP05j2ErsjmcR1PPS5XJwWmA5eBcRPlbr_2S0JQWOwD.NiV5juPlRmzXGi_FGyZQUZP9hb1QV VE0I- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Wed, 22 Feb 2023 03:31:18 +0000 Received: by hermes--production-ne1-746bc6c6c4-b6fc9 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 6e098ad77d43fb6e12f2ab8fb68e4c99; Wed, 22 Feb 2023 03:31:12 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: [analysis] 13.2 BETA2: how do debug META_MODE? From: Mark Millard In-Reply-To: Date: Tue, 21 Feb 2023 19:31:01 -0800 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: 7bit Message-Id: References: To: Peter X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4PM1sN5tBPz3pbR X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Feb 21, 2023, at 18:32, Peter wrote: > It appears as my issue goes away by itself at the fourth subsequent > build. > >> # cd /usr/src/ >> # env WITH_META_MODE=yes make buildworld >> # env WITH_META_MODE=yes make installworld >> # env WITH_META_MODE=yes make buildworld (again #0) >> ## no more rebuilds below? >> # env WITH_META_MODE=yes make buildworld (again #1) >> # env WITH_META_MODE=yes make buildworld (again #2) >> >> "again #0" will rebuild llvm/clang. The other two "again"s >> will not. > > Strangely, I observed rather the opposite: the issue was with those > nodes that do *not* get installed. Those that do get installed, they > behaved as expected. My older notes from 2021 were better at being explicit about the initial context assumed: QUOTE Given an already built, installed and booted system version, I've noted a big difference for META_MODE in 2 rebuild contexts (no source updates involved) END QUOTE In other words, all the build activity was rebuild-same-version activity or reinstall-same-version activity in my example sequence. So I expect that the attempted comparison happens to not have a matching context to make the comparison with. > Details: > > When starting a build, there are some programs in > /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ from the last build. > These might be suitable for the new build (or they might not, who > knows) - but then, very early in the buildworld, many of these files > are copied from the running base system, with their mtime preserved. > So now these files do not have an mtime of the last build, but of the > last *install* of this running instance. (And that may be too late.) > This is done here - and I have no idea how these files are selected: > >> -------------------------------------------------------------- >>>>> Rebuilding the temporary build tree >> -------------------------------------------------------------- > [...] >> Building /usr/obj/usr/src/amd64.amd64/tools/build/host-symlinks >> Linking host tools into /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin > > Over all, /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ tends to become > a zoo: some files are copied from the base system, some are installed > during the build, some are not replaced at all. > > I tried to figure what exactly is happening, with the example of > the make-roken + roken.h. > I start the upgrade to 13.2 with an empty obj tree (because after > switching and rebasing branches, my mtimes are recreated from the > latest commit - and they will certainly not align with the old ones). > > In the first build the binaries in the obj tree have to be created. > But since the running system is still 13.1, these binaries get all > stamped for 13.1. > > In the second build -now running 13.2- META_MODE decides that the > mtimes are now fine and does not rebuild. > META_MODE has no notion of the currently running OS version, it only > considers mtimes and cannot detect that these dependencies are stamped > with an old OS version. In terms of your example, in case the earlier was unclear: In my test sequence, 13.2 would have already been built, installed, and booted before the buildworld/installworld sequence that I listed. So my notes do not match your full sequence. They are more of a rebuild-already-existing sequence to analyze the behavior just for that simpler context than a realistic overall normal-build sequence. Having a installworld is only there to see what differences it makes compared to later not having such between such rebuild-already-existing runs. The isntallworld was an example of reinstall-same-version-as-already-running. > In the third build META_MODE finds some mtimes obsolete again, and > does rebuild toolchain binaries. This would not have any effect > because "install" would not copy them to the bin-dir if the same > binaries are already there. But in this scenario the old binaries > are from 13.1, so they are different, and "install" copies the new > ones in - and now everything that somehow depends on them will also > rebuild. > > Finally in the fourth build everything appears to be fine. > > Conclusion: > For a safe upgrade (specifically for a major version change) it is > not so much necessary to delete the obj tree before the first build, > but rather *after* the first build, i.e. after the base system has > been upgraded. > > Some timings: > > The base gets installed into a clean DESTDIR and used for the next > pass. The obj-trees are individually kept. > > Initial (obj trees deleted) > ( 4 vcore) > 230217231308.base.pass1.sst: real 9339.64 base w/ kernels > 230217231308.base.pass2.sst: real 7982.69 > 230217231308.admn.pass1.jail.sst:real 9346.90 jail w/ compiler > 230217231308.admn.pass2.jail.sst:real 5460.16 > 230217231308.data.pass1.jail.sst:real 4094.04 jail w/o compiler > 230217231308.data.pass2.jail.sst:real 143.39 > 230217231308.iamk.pass1.jail.sst:real 8050.27 jail w/ compiler > 230217231308.iamk.pass2.jail.sst:real 5226.32 > 230217231308.oper.pass1.jail.sst:real 2910.28 jail w/o compiler > 230217231308.oper.pass2.jail.sst:real 92.05 > 230217231308.rail.pass1.jail.sst:real 3236.29 jail w/o compiler > 230217231308.rail.pass2.jail.sst:real 99.49 > 230217231308.tele.pass1.jail.sst:real 3170.34 jail w/o compiler > 230217231308.tele.pass2.jail.sst:real 180.65 > > pass3 > (10 vcore) > 230222000242.base.std.sst: real 1162.80 base w/ kernels > 230222000242.admn.std.jail.sst:real 1759.15 jail w/ compiler > 230222000242.data.std.jail.sst:real 155.54 jail w/o compiler > 230222000242.iamk.std.jail.sst:real 1715.07 jail w/ compiler > 230222000242.oper.std.jail.sst:real 149.51 jail w/o compiler > 230222000242.rail.std.jail.sst:real 151.73 jail w/o compiler > 230222000242.tele.std.jail.sst:real 150.52 jail w/o compiler > > pass4 > (10 vcore) > 230222021535.edge.std.sst: real 1018.79 base w/ kernels > 230222021535.admn.std.jail.sst:real 101.61 jail w/ compiler > 230222021535.data.std.jail.sst:real 67.47 jail w/o compiler > 230222021535.iamk.std.jail.sst:real 100.91 jail w/ compiler > 230222021535.oper.std.jail.sst:real 66.52 jail w/o compiler > 230222021535.rail.std.jail.sst:real 68.00 jail w/o compiler > 230222021535.tele.std.jail.sst:real 66.54 jail w/o compiler === Mark Millard marklmi at yahoo.com