FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes)

[Simon J. Gerraty]

Mark Millard via freebsd-current <freebsd-current at FreeBSD.org> wrote:
> >> Given an already built, installed and booted system version, I've
> >> noted a big difference for META_MODE in 2 rebuild contexts (no
> >> source updates involved):
> >>
> >> *) Prior buildworld buildkernel, installkernel, installworld, boot
> >>  presumed before (A) and before (B) below.

Yes installworld is going to cause problems unless you tell meta mode to
ignore everything outside of the src/obj tree.
But even that isn't enough as you note below.

> >> So I used make -dM for the commented buildworld buildkernel lines, logging
> >> the build output and later diff'ing them.
> >>
> >> Result that I noticed? Lots of lines uniquely from (B)'s case, ending with
> >> one of:
> >>
> >> file '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/awk' is newer than the target...

Yes.  That would be expected.
I think Bryan added some knobs to mitigate some of this.

grep -n META.*IGNORE share/mk/*mk

might be instructive.

> >> Many/most/all(?) of these seem to me to be unlikely to actually need to
> >> contribute to what needs to be rebuilt (just based on being newer). So
> >> the option to ignore (some of?) them could be useful in making META_MODE
> >> builds quicker.

Yes on all counts.  That's why bmake provides a number of knobs to
ignore such things.
They are listed in the man page in increasing order of expense.

One of the risks of the sort of introspection meta mode does, is delving
too deep (like the dwarfs ;-)

The .MAKE.META.IGNORE_* and .MAKE.META.BAILIWICK variables help to
contain the potential insanity.

> > The following from one of the .meta files makes the point that rm use
> > in the example is unlikely to be important to needing to rebuild,
> > despite it actually causing a file rebuild. Nor is the specific echo
> > command listed relevant. Only the "ar" command is:
> >
> > # Meta data file /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++/libc++.a.meta
> > CMD @echo building static c++ library
> > CMD @rm -f libc++.a
> > CMD ar -crsD libc++.a algorithm.o any.o atomic.o barrier.o bind.o

> > -- filemon acquired metadata --
> > # filemon version 5
> > # Target pid 22471
> > # Start 1611359217.214996
> > V 5
> > E 22961 /bin/sh
> > R 22961 /etc/libmap.conf
> > R 22961 /var/run/ld-elf.so.hints
> > R 22961 /lib/libedit.so.7
> > R 22961 /lib/libc.so.7
> > R 22961 /lib/libncursesw.so.9
> > R 22961 /usr/share/locale/C.UTF-8/LC_CTYPE
> > F 22961 22962
> > E 22962 /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/rm

> > The timestamp on . . ./tmp/legacy/usr/sbin/rm is not
> > actually relevant to if libc++.a needs to be rebuilt.

True. 
If there is nothing under .../tmp/legacy that should be counted you can
just:

.MAKE.META_IGNORE_PATHS += that path

which would be the cheapest option.

If however there are things that should be considered you'd have to
use a much more specific list of .MAKE.META_IGNORE_PATHS or
perhaps something like:

.MAKE.META.IGNORE_PATTERNS += */rm

would ignore an rm binary regardless of where found.

worst case you might need something like:

# realpath
.MAKE.META.IGNORE_FILTER += tA
# ignore anything here
.MAKE.META.IGNORE_FILTER += N*/tmp/legacy/usr/bin/*

Unlike .MAKE.META.IGNORE_PATTERNS which is a list of patterns to match
the goal with .MAKE.META.IGNORE_FILTER  is to reduced to an empty string
paths to be ignored.

> > Of course, the structure also point out the judgment
> > is specific to understanding the sequence of CMD's
> > listed above. Only a hack of ignoring, not recording,
> > or commenting out the filemon lines ending in
> > /tmp/legacy/usr/sbin/rm would seem to avoid the @rm
> > handling issue. Such might well have its own risks.

No hacking needed, there are a range of knobs to help.

> Just for reference for more about the sequencing involved:
> 
> Looks like in my example various . . ./tmp/legacy/. . ./*bin/
> actually are links to files in:
> 
> /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/bin/
> 
> and the later after-install buildworld "Rebuilding the
> temporary build tree" step leads to the updated dates for
> files in that area, updated via the code that reports:
> 
> Linking host tools into /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/bin
>
> So the prior installworld leaves updated dates and the
> linking to those installed files makes
> . . ./tmp/legacy/. . ./*bin/* have the newer dates show
> up for the legacy paths as well.
> 
> In turn the dependency tracking via META_MODE uses the new
> dates on . . ./tmp/legacy/. . ./*bin/* files to cause
> rebuilds of more materials than if installworld had not
> been done.
> 
> It is not obvious if Bryan D. would find the effort to avoid
> this worthwhile for the contexts that drive FreeBSD's build
> environment choices.

For people that  use installworld and also want META_MODE
it would seem some additional IGNORE work may be beneficial.

--sjg