Re: 13.2 BETA2: how do debug META_MODE?

On Feb 21, 2023, at 19:11, Peter <pmc@citylink.dinoex.sub.org> wrote:

> On Tue, Feb 21, 2023 at 06:44:09PM -0800, Mark Millard wrote:
> ! On Feb 21, 2023, at 18:10, Peter <pmc@citylink.dinoex.sub.org> wrote:
> ! 
> ! > On Tue, Feb 21, 2023 at 11:56:13AM -0800, Mark Millard wrote:
> ! > ! On Feb 21, 2023, at 04:55, Peter <pmc@citylink.dinoex.sub.org> wrote:
> ! > ! 
> ! > ! > ! # cd /usr/src/
> ! > ! > ! # env WITH_META_MODE=yes make buildworld
> ! > ! > ! # env WITH_META_MODE=yes make installworld
> ! > ! > ! # env WITH_META_MODE=yes make buildworld (again #0)
> ! > ! > ! ## no more rebuilds below?
> ! > ! > ! # env WITH_META_MODE=yes make buildworld (again #1)
> ! > ! > ! # env WITH_META_MODE=yes make buildworld (again #2)
> ! > ! > 
> ! > ! > But what is the difference between #0 and #1?
> ! > ! 
> ! > ! awk, cp, ln, rm, sed, and many more from
> ! > ! . . ./tmp/legacy/usr/sbin/have new dates
> ! > ! for rebuilds after installworld (that targets
> ! > ! the running system). Not true for #1 and #2.
> ! > ! 
> ! > ! The dates on these tools being more recent than
> ! > ! the files that they were involved in producing
> ! > ! leads to rebuilding those files. That in turn
> ! > ! leads to other files being rebuilt.
> ! > ! 
> ! > ! make with -dM reports the likes of:
> ! > ! 
> ! > !    file '. . ./tmp/legacy/usr/sbin/awk' is newer than the target...
> ! > ! 
> ! > ! explicitly as it goes. As I remember tmp/legacy/usr/sbin/
> ! > ! was always part of the path for what I found.
> ! > 
> ! > Mark, thanks a lot for the proper input at the right time!
> ! > 
> ! > This put me on the right track and I mananged to analyze and
> ! > understand what is actually happening.
> ! > 
> ! > It looks like my issue does resolve itself somehow, and things
> ! > start  to behave as expected again after four builds.
> ! 
> ! Intersting.
> ! 
> ! > ! I did not do the analysis of how (e.g.) tmp/legacy/usr/sbin/awk
> ! > ! ended up being newer than such a target and, so, causing a
> ! > ! rebuild of that target. I was going the direction: that
> ! > ! it is newer really is unlikely to justify the rebuild for
> ! > ! the target(s) in question. The other direction about how
> ! > ! it got to be newer is also relevant.
> ! > 
> ! > I have now analyzed some parts of it. META_MODE typically finds some
> ! > build-tools to rebuild, but then if the result is not different
> ! > from what was there before, then "install" will not copy it to the
> ! > bin-dir, and so the avalanche gets usually avoided.
> ! > 
> ! 
> ! The implication is that "install -C" is in use, quoting the
> ! man page:
> ! 
> !      -C      Copy the file.  If the target file already exists and the files
> !              are the same, then do not change the modification time of the
> !              target.  If the target's file flags and mode need not to be
> !              changed, the target's inode change time is also unchanged.
> ! 
> !      -c      Copy the file.  This is actually the default.  The -c option is
> !              only included for backwards compatibility.
> ! 
> ! -C might have more of an effect in a reproducible-build
> ! style build process than on a non-reproducible-build
> ! style one.
> 
> Yepp. "install -p" is used, see /usr/src/tools/install.sh
> 

The code for the _bootstap_tools_links uses "cp -pf",
not install, to establish part of . . ./tmp/legacy/bin/ .
(Note: . . ./tmp/legacy/sbin -> ../bin so is a via a
symbolic link.) Before the "cp -pf" there is a "rm -f"
deleting the target file before the copy: the prior
file in . . ./tmp/legacy/bin/ is never directly
preserved. (The new copy might still be identical
to the old one: the source path one might happen to
be identical as well.)

# Link the tools that we need for building but don't need to bootstrap because
# the host version is known to be compatible into ${WORLDTMP}/legacy
# We do this before building any of the bootstrap tools in case they depend on
# the presence of any of the links (e.g. as m4/lex/awk)
${_bt}-links: .PHONY

.for _tool in ${_bootstrap_tools_links}
${_bt}-link-${_tool}: .PHONY
        @rm -f "${WORLDTMP}/legacy/bin/${_tool}"; \
        source_path=`which ${_tool}`; \
        if [ ! -e "$${source_path}" ] ; then \
                echo "Cannot find host tool '${_tool}'"; false; \
        fi; \
        cp -pf "$${source_path}" "${WORLDTMP}/legacy/bin/${_tool}"
${_bt}-links: ${_bt}-link-${_tool}
.endfor

Note: This is for the !defined(BOOTSTRAP_ALL_TOOLS) case.
Note: the code uses the abbreviation: _bt=            _bootstrap-tools

_bootstrap_tools_links is built mostly in terms of
_basic_bootstrap_tools and _basic_bootstrap_tools_multilink from
earlier logic. For reference, showing what ends up handled
this way:

# grep -r "_bootstrap_tools" /usr/main-src/Makefile* /usr/main-src/share/ | more
/usr/main-src/Makefile.inc1:# _bootstrap_tools_links variable.
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=m4 lex
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=mtree
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=cat
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=crunchide
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=crunchgen
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=mkimg
/usr/main-src/Makefile.inc1:_kerberos5_bootstrap_tools= \
/usr/main-src/Makefile.inc1:.ORDER: ${_kerberos5_bootstrap_tools:C/^/${_bt}-/g}
/usr/main-src/Makefile.inc1:.for _tool in ${_kerberos5_bootstrap_tools}
/usr/main-src/Makefile.inc1:# The tools listed in _basic_bootstrap_tools will generally not be
/usr/main-src/Makefile.inc1:# case we use the _basic_bootstrap_tools_multilink variable which is a list of
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools_multilink=usr.bin/grep grep,egrep,fgrep
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools_multilink+=bin/test test,[
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.bin/cut bin/expr usr.bin/gencat usr.bin/join \
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools_multilink+=usr.bin/awk awk,nawk
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.bin/file2c
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools_multilink+=usr.bin/bintrans uuencode,uudecode
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.bin/xargs
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.bin/cap_mkdb
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.sbin/services_mkdb usr.sbin/pwd_mkdb
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.bin/ldd
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=bin/chflags
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=sysctl
/usr/main-src/Makefile.inc1:_other_bootstrap_tools+=tools/build/cross-build/fake_chflags
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.bin/mkfifo
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.bin/jot
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=sbin/md5
/usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=usr.sbin/tzsetup
/usr/main-src/Makefile.inc1:_other_bootstrap_tools+=${_basic_bootstrap_tools}
/usr/main-src/Makefile.inc1:.for _subdir _links in ${_basic_bootstrap_tools_multilink}
/usr/main-src/Makefile.inc1:_other_bootstrap_tools+=${_subdir}
/usr/main-src/Makefile.inc1:_other_bootstrap_tools+=usr.bin/bmake
/usr/main-src/Makefile.inc1:_other_bootstrap_tools+=lib/libbz2
/usr/main-src/Makefile.inc1:_other_bootstrap_tools+=lib/libz
/usr/main-src/Makefile.inc1:_other_bootstrap_tools+=lib/libcrypt
/usr/main-src/Makefile.inc1:# All tools in _basic_bootstrap_tools have the same name as the subdirectory
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=${_basic_bootstrap_tools:T}
/usr/main-src/Makefile.inc1:.for _subdir _links in ${_basic_bootstrap_tools_multilink}
/usr/main-src/Makefile.inc1:_bootstrap_tools_links+=${_links:S/,/ /g}
/usr/main-src/Makefile.inc1:.for _tool in ${_bootstrap_tools_links}
/usr/main-src/Makefile.inc1:    ${_kerberos5_bootstrap_tools} \
/usr/main-src/Makefile.inc1:    ${_other_bootstrap_tools} \

What the prior installworld did for the analogous files is
a separate issue.

===
Mark Millard
marklmi at yahoo.com