Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes)

On Feb 22, 2023, at 19:47, Mark Millard <marklmi@yahoo.com> wrote:

> On Feb 22, 2023, at 17:18, Simon J. Gerraty <sjg@juniper.net> wrote:
> 
>> Mark Millard <marklmi@yahoo.com> wrote:
>> 
>>> Thanks for the information.
>>> 
>>>> strings `which bmake` | grep META.IGNORE
>>>> .MAKE.META.IGNORE_PATHS
>>>> .MAKE.META.IGNORE_PATTERNS
>>>> ${.MAKE.META.IGNORE_PATHS:O:u:tA}
>>> 
>>> The -dM output's "is newer than the target" lines
>>> show the path from before the above transformation.
>>> (The :tA results possibly could use another
>>> sort/uniq sequence for the realpath results?)
>> 
>> That indicates the above IGNOREs are not working.
>> 
>>> I've been pondering things because, so far, my
>>> attempts to experiment with this has failed to make
>>> the -dM output lines for the paths go away and it
>>> still does the related build activity. I've been
>>> trying the likes of:
>>> 
>>> .for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide dd egrep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp mv patch realpath rm sed sh touch truncate uudecode uuencode
>>> xargs
>> 
>> Is there anything under ${OBJTOP}/tmp that you don't want to ignore?
> 
> More than just _bootstrap_tools_links entries end up in
> ${WORLDTMP}/legacy/bin/  (so in ${WORLDTMP}/legacy/sbin/
> via the symbolic link pointing to ${WORLDTMP}/legacy/bin/ ).
> So: yes.
> 
> Also, OBJTOP is not constant over all the parts of
> buildworld buildkernel . Having the late-substitution
> form of notation ${OBJTOP} might not be appropriate
> for the content of .MAKE.META.IGNORE_PATHS .
> 
> I'm trying to figure out if there is a stable way of
> getting a path that would not suffer variability
> via late substitution. 
> 
>> Otherwise you could simply use
>> 
>> .MAKE.META.IGNORE_PATHS+= ${OBJTOP}/tmp/
> 
> (Ignoring the variability of OBJTOP issue . . .)
> 
> I do not expect that would work: ignoring things
> it likely should not.
> 
> Also, I'd rather grow a smaller set of ignores
> gradually to make it easier to detect if an
> addition starts causing a problem and can be
> backed out. Starting with everything ignored
> would make things much harder to figure out
> when ignoring creates a problem.
> 
>> You might need ${OBJTOP:tA}/tmp/
>> or both.
>> 
>>> .MAKE.META.IGNORE_PATHS+= ${OBJTOP}/tmp/legacy/usr/sbin/${ignore_legacy_tool}
>>> .endfor
>>> .for ignore_other_tool in ctfconvert objcopy nm
>>> .MAKE.META.IGNORE_PATHS+= ${OBJTOP}/tmp/usr/bin/${ignore_other_tool}
>>> .endfor
>>> 
>>> in what I use for make.conf via:
>>> 
>>> __MAKE_CONF=/usr/home/root/src.configs/make.conf
>>> 
>>> It is using paths that match the -dM output lines ( sbin
>>> use despite sbin -> ../bin being a symbolic link).
>>> 
>>> Note: WORLDTMP is not defined that early, thus the ${OBJTOP}/tmp
>>> use.
>>> 
>>> -V.MAKE.META.IGNORE_PATHS is showing the paths I would
>>> expect, matching the -dM lines.
>> 
>> Do you have example?
>> I really need to add some unit-tests for these...
> 
> You may want to wait while I see if I can come up with
> a better example context to show things. I only just
> noticed the late-substitution potential issue and
> started looking for a way to avoid it, for example.
> (Either a value that does not vary or a form of
> causing up-front substitutions in my make.conf .)
> 

I got some useful information about one type of context
that is odd and creates a far mount of "is newer than
the target" notices.

This is an example of something that always "rebuilds"
via a . . ./sbin/realpath "is newer":

/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/machine.meta: 23: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/realpath' is newer than the target...
Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/machine

It turns out that the .meta file is for a symbolic link:

# ls -Tld /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/machine
lrwxr-xr-x  1 root  wheel  31 Feb 22 20:19:27 2023 /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/machine -> /usr/main-src/sys/amd64/include

. . ./sys/modules/*/machine being a symbolic link to a directory is normal.

It turns out that . . ./sbin/ln "is newer" examples are
tied to a similar issue:

/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/awk/awkgram.tab.h.meta: 12: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/ln' is newer than the target...
Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/awk/awkgram.tab.h

But . . .

# ls -Tld /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/awk/awkgram.tab.h
lrwxr-xr-x  1 root  wheel  9 Feb 22 20:18:24 2023 /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/awk/awkgram.tab.h -> awkgram.h

The .meta file is for a symbolic link, to a file for this
type of context, not to a directory. (All the examples
happen for what I'm looking at happen to be symbolic
links to header files.)

It appears that for symbolic links being the target, META_MODE does
not respect .MAKE.META.IGNORE_PATHS .

===
Mark Millard
marklmi at yahoo.com