Re: [analysis] 13.2 BETA2: how do debug META_MODE?

On Feb 21, 2023, at 18:32, Peter <pmc@citylink.dinoex.sub.org> wrote:

> It appears as my issue goes away by itself at the fourth subsequent
> build.
> 
>> # cd /usr/src/
>> # env WITH_META_MODE=yes make buildworld
>> # env WITH_META_MODE=yes make installworld
>> # env WITH_META_MODE=yes make buildworld (again #0)
>> ## no more rebuilds below?
>> # env WITH_META_MODE=yes make buildworld (again #1)
>> # env WITH_META_MODE=yes make buildworld (again #2)
>> 
>> "again #0" will rebuild llvm/clang. The other two "again"s
>> will not.
> 
> Strangely, I observed rather the opposite: the issue was with those
> nodes that do *not* get installed. Those that do get installed, they
> behaved as expected.

My older notes from 2021 were better at being explicit
about the initial context assumed:

QUOTE
Given an already built, installed and booted system version, I've
noted a big difference for META_MODE in 2 rebuild contexts (no
source updates involved)
END QUOTE

In other words, all the build activity was rebuild-same-version
activity or reinstall-same-version activity in my example
sequence.

So I expect that the attempted comparison happens to not have a
matching context to make the comparison with.

> Details:
> 
> When starting a build, there are some programs in
> /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ from the last build.
> These might be suitable for the new build (or they might not, who
> knows) - but then, very early in the buildworld, many of these files
> are copied from the running base system, with their mtime preserved.
> So now these files do not have an mtime of the last build, but of the
> last *install* of this running instance. (And that may be too late.)
> This is done here - and I have no idea how these files are selected:
> 
>> --------------------------------------------------------------
>>>>> Rebuilding the temporary build tree
>> --------------------------------------------------------------
> [...]
>> Building /usr/obj/usr/src/amd64.amd64/tools/build/host-symlinks
>> Linking host tools into /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin
> 
> Over all, /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ tends to become
> a zoo: some files are copied from the base system, some are installed
> during the build, some are not replaced at all.
> 
> I tried to figure what exactly is happening, with the example of
> the make-roken + roken.h.
> I start the upgrade to 13.2 with an empty obj tree (because after
> switching and rebasing branches, my mtimes are recreated from the
> latest commit - and they will certainly not align with the old ones).
> 
> In the first build the binaries in the obj tree have to be created.
> But since the running system is still 13.1, these binaries get all
> stamped for 13.1.
> 
> In the second build -now running 13.2- META_MODE decides that the
> mtimes are now fine and does not rebuild.
> META_MODE has no notion of the currently running OS version, it only
> considers mtimes and cannot detect that these dependencies are stamped
> with an old OS version.

In terms of your example, in case the earlier was unclear:

In my test sequence, 13.2 would have already been built, installed,
and booted before the buildworld/installworld sequence that I
listed. So my notes do not match your full sequence. They are more
of a rebuild-already-existing sequence to analyze the behavior
just for that simpler context than a realistic overall normal-build
sequence. Having a installworld is only there to see what
differences it makes compared to later not having such between
such rebuild-already-existing runs. The isntallworld was an
example of reinstall-same-version-as-already-running.

> In the third build META_MODE finds some mtimes obsolete again, and
> does rebuild toolchain binaries. This would not have any effect
> because "install" would not copy them to the bin-dir if the same
> binaries are already there. But in this scenario the old binaries
> are from 13.1, so they are different, and "install" copies the new
> ones in - and now everything that somehow depends on them will also
> rebuild.
> 
> Finally in the fourth build everything appears to be fine.
> 
> Conclusion:
> For a safe upgrade (specifically for a major version change) it is
> not so much necessary to delete the obj tree before the first build,
> but rather *after* the first build, i.e. after the base system has
> been upgraded.
> 
> Some timings:
> 
> The base gets installed into a clean DESTDIR and used for the next
> pass. The obj-trees are individually kept.
> 
> Initial (obj trees deleted)
> ( 4 vcore)
> 230217231308.base.pass1.sst:     real 9339.64 base w/ kernels
> 230217231308.base.pass2.sst:     real 7982.69
> 230217231308.admn.pass1.jail.sst:real 9346.90 jail w/ compiler
> 230217231308.admn.pass2.jail.sst:real 5460.16
> 230217231308.data.pass1.jail.sst:real 4094.04 jail w/o compiler
> 230217231308.data.pass2.jail.sst:real 143.39
> 230217231308.iamk.pass1.jail.sst:real 8050.27 jail w/ compiler
> 230217231308.iamk.pass2.jail.sst:real 5226.32
> 230217231308.oper.pass1.jail.sst:real 2910.28 jail w/o compiler
> 230217231308.oper.pass2.jail.sst:real 92.05
> 230217231308.rail.pass1.jail.sst:real 3236.29 jail w/o compiler
> 230217231308.rail.pass2.jail.sst:real 99.49
> 230217231308.tele.pass1.jail.sst:real 3170.34 jail w/o compiler
> 230217231308.tele.pass2.jail.sst:real 180.65
> 
> pass3
> (10 vcore)
> 230222000242.base.std.sst:     real 1162.80 base w/ kernels
> 230222000242.admn.std.jail.sst:real 1759.15 jail w/ compiler
> 230222000242.data.std.jail.sst:real 155.54 jail w/o compiler
> 230222000242.iamk.std.jail.sst:real 1715.07 jail w/ compiler
> 230222000242.oper.std.jail.sst:real 149.51 jail w/o compiler
> 230222000242.rail.std.jail.sst:real 151.73 jail w/o compiler
> 230222000242.tele.std.jail.sst:real 150.52 jail w/o compiler
> 
> pass4
> (10 vcore)
> 230222021535.edge.std.sst:     real 1018.79 base w/ kernels
> 230222021535.admn.std.jail.sst:real 101.61 jail w/ compiler
> 230222021535.data.std.jail.sst:real 67.47 jail w/o compiler
> 230222021535.iamk.std.jail.sst:real 100.91 jail w/ compiler
> 230222021535.oper.std.jail.sst:real 66.52 jail w/o compiler
> 230222021535.rail.std.jail.sst:real 68.00 jail w/o compiler
> 230222021535.tele.std.jail.sst:real 66.54 jail w/o compiler

===
Mark Millard
marklmi at yahoo.com