From nobody Wed Feb 22 04:51:59 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PM3fr4PJRz3sGRL for ; Wed, 22 Feb 2023 04:52:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-21.consmr.mail.gq1.yahoo.com (sonic314-21.consmr.mail.gq1.yahoo.com [98.137.69.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PM3fr1p68z3wY5 for ; Wed, 22 Feb 2023 04:52:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677041537; bh=TkB0mxll72gJ3VQ9gKn81dOrnLA3FyQCeFtowyVcSAA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=udpKd8GmWu5aWfWlKOoQpi0G9SQ8g/K+8qrq3ax2IO4X0Zb6i+WTeDPsFpwNLAkhWlyWZo2l6X/hsOeICC3QbZKmQr2YptQjIX/En4vrqQNLJuUMYyga9JltivasUBw6N3LrCIXs4nJMa5kPpWcFaGoowgfQUx3+JI8y7bgLnVzXp4CAvWBCTUr86/L4YmzUR60H05ck5levszeGbtDZL3Cpr6kyiEn8UNPCiF0c1dDDH5eypWB4QmUZ/y7Z25OkcruNGm0KfYh+U54dhVLw4hXc5LM/XI7JGilWe448pp+FdbZzClp4LHFfD4qM/pvwoAXcM4iusNn2++lUNy/XpA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677041537; bh=XCppq19LewiarMWUeKSFFJAiMMvz2+EgxZwmWCdysLJ=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=uB4/tKo8jPqbW1nvgp8UBypijOJhrlkoVq17Z+KCT/Fp1tdvuhoSjqVmSUSmaha9dJZk1h90AAFXoWGbqdViipTqaN+q10nCoFtdMG6z3fyqcTO59Jcrsvvjg3vv0Y+sfIdyGP7FiobOUXtZsTVHbK2EgggFFHzDEXcX+/waARL0Gn7eShiS4QEYqDwLlqd9NE8qWc/LjFhJfOhy3nfiMTF+VfFS49ZpSwivKoOywA5nBqT/Vbr30FGpjR+xFiORZ0IyCwkqSp1d9SRs+CkWxyGqfKuK5aGmZpU4DQomBbPuVJ50nRe3y6ixOzJNvaCLVhSDLzB0tukKeBWaKQCEaA== X-YMail-OSG: WoPKsxQVM1lqtg8.8XvTMcWlXlLm51LAK24DHH3j4l0Bu415K_GxRmTpw8FQ9JK JDkKMS.mvA8LsGA4iP6t3VLq74QZM5zRnsPnOtLGHdG4Sn1QkDNhViNpX3oB4U.Cp.pYpfhbXPeC CrGbhSEm6d75Qk2595sZj6CtcldS0Msxy6BsRMCVBkarZ4kQk6qtjBy5JkiElPY9TUNXejEXgGp8 9ma2OQHern22mS18Eo_31kKsAZiVdDC.5pimmU2dkDw0qSVowJO3eNhF0QI55wSCqCv_JG05h9la Lyhj6x2vmEmya_wVU8PQ_pMVAdSwy_Hq1.70OEB5Ou4HVwTFmGU9EhXA97zjCdDOr5er17bAt20U sZOt_sW2T8tcBP0KPNYRPod4TJaqgM_kI4MQdlPYZp963ksWiQCFO4IazWMk4psWdUkYTVSxhogA vyijSNWZpbuQr0jUnHzxNSqmgeXTmr.jZaZ3.GSeticy4aOn8kT5kyRXG_uOilBortMMT86lOvj6 wfcp9AJUlZuY8gQcgFykpfMgBLBMi3Jd808B7jl.n62JLDd1OM6stfawaCZAhdTNNQU1la6CtruX PTl3FsxPFN865OQ6tIL7TEpkRl0hq32tcSlsTb6gZLRVCFlfgZbpB_vq72RjE0bYVusSMo.SIwjD H4vauWLMWZ2qxsEoHdh8wHUf1XbtJd1mKEgJwm54oheQbqDhzIu7rIqUJhomo7Jn2XgDULHY2vmC 5B_7iPmPQZMVGTHrfAz1ccq2CqxAt2jEdXwLHyXQ_xDM.ICaDGMcp2RXzxY8sVYVqTTX.sTsxPto LV.H3PxG8l9GYcMh7.rQfJVSQl16Zrd1IUDPDBavjclJHlgpwtjfVd3DDp7pxBdwzz8Y4sFGGr5V IOJ422dFCOo2vJQ4itVbrcMuLTULZM5pWT0o9lOJSDQ.AhOKGEn66ptRIhC6APc9A1_B5ltjAvtg wBdsqe4VxaaOv9p6bLuxMG77UZsUS419qO9M6HD1ogDnkr2QxL5lK03bF3azN5UPXh8vAXZDY0vn .n.TmIQBPx98K_yrmTWeLexeKaMQv0ScruXYmVIt9FIxL6d1VtXq9fO98HWdakYtxy3KDJqQjaTT W.17TClK8MQ3BdvHpHdy9w3WLLsHb9lFHUDzOi75MZDnJwUVc5XhIzpDBkW6U4csBmlND0TUbpMr 2BPIBh4Aw3iQssuR3.ZGMR5SfniUbv9kxGIJJ31KVwiENqZjQb2FGfaZ8Jy5BH2Thr6.edGcriND 2SQQ1SjldG7mZ6hZJz0dhvri3CIRvNKcgv0K5__MkkTGMgg6oSuNj31hVj0bSTq_hVk6Ik4Y3x3J MGtdkIacmjzw8jl6JEuFhKeJBkHliyqwGaGyxDRu0UhZU2f_k7j6M0wWaFesR4p0xhh2f.9MowT7 8enCkDLHIW4.SJZVrwGLVNCN5KVqVca29HFDCA2mbmG3KiLTbiDd6quo_bCwcPLBjOIizZ8.Zw4Q URvOJ71K.TyamQXuH8yHrVcUeQAlUOx0JvhaigDpiZ0TddvcX0ttE2Ycw5Yow.jswVT8rGtxDelb uCM_k300su_Vd9ZmdUfI_yqoSJm6HAbPs6pGjAlfL4LoHUAr1MkYDH2RinLn5ZW0TEBMgYLBtpSB dO945tnTA3Asditk9xVZmyM0lHZquoh1f1e0q6GWpLbc0ypp0LXu0UYeCjL4.wMpX8B4hYpjekyS VpmXb_K1FIXFH3K_Zb3tcUBeZf.qdb2yz3pqrjwSSkS1W5zb1G5FO0TOZ1ULcTBQeF6ZV2Y0WwP8 605qI6bkRp5X5Fy1HJBTmtnkwswam6L6zeR4Z6Z1aGCIvnIUEGNlcAfwtatQbmQ14gcFfwnlRWZU F64S.N0Id4pSYzQYtZljUGgGo5HrEqJHITwUXEFA.7Tn.aps9C4XpVklgvePEkZHp4zPY2M8RcUA .mtNzmjqv9sSiXgEtzP.3wpg8_iuCEha8wrsudwzQqcUNPWvw79UhMiTO1WpHOls8SW7hqxKtCf7 sddiRTYPXHq9etI.ab3WsCFKHUuDFMglCjW8_oCmrP3yG7PX.iYJDGuKMK8w.VPxei24mMgpedXk 6emDU5TskIy0Jqz1xuCnuVITGtl2MoCJbaTdy.zooTVbisk8P6oYjeRxNZ.akr6RDSLNXWeeXPE9 .RU74xX03_6d1cYw1ySj6LvlqNcT1lWi606kc4PQkKA7Ymcqc7l25iKeYNhjfQ8.vJNeePcCznLa 9V_.0juTXZygnaWIzQDWIh0oNHNHMMpTXHSkAzVHUnKACcXFuusHZXXVheB.twx1r48T9xUVaY5E brA4- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Wed, 22 Feb 2023 04:52:17 +0000 Received: by hermes--production-ne1-746bc6c6c4-7ksjp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ab387727fd17f95e1fa460a1a5fea392; Wed, 22 Feb 2023 04:52:10 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: 13.2 BETA2: how do debug META_MODE? From: Mark Millard In-Reply-To: Date: Tue, 21 Feb 2023 20:51:59 -0800 Cc: FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: References: <41B536B0-DA66-449E-96BB-E11A8750471A.ref@yahoo.com> <41B536B0-DA66-449E-96BB-E11A8750471A@yahoo.com> To: Peter X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4PM3fr1p68z3wY5 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Feb 21, 2023, at 19:11, Peter wrote: > On Tue, Feb 21, 2023 at 06:44:09PM -0800, Mark Millard wrote: > ! On Feb 21, 2023, at 18:10, Peter = wrote: > !=20 > ! > On Tue, Feb 21, 2023 at 11:56:13AM -0800, Mark Millard wrote: > ! > ! On Feb 21, 2023, at 04:55, Peter = wrote: > ! > !=20 > ! > ! > ! # cd /usr/src/ > ! > ! > ! # env WITH_META_MODE=3Dyes make buildworld > ! > ! > ! # env WITH_META_MODE=3Dyes make installworld > ! > ! > ! # env WITH_META_MODE=3Dyes make buildworld (again #0) > ! > ! > ! ## no more rebuilds below? > ! > ! > ! # env WITH_META_MODE=3Dyes make buildworld (again #1) > ! > ! > ! # env WITH_META_MODE=3Dyes make buildworld (again #2) > ! > ! >=20 > ! > ! > But what is the difference between #0 and #1? > ! > !=20 > ! > ! awk, cp, ln, rm, sed, and many more from > ! > ! . . ./tmp/legacy/usr/sbin/have new dates > ! > ! for rebuilds after installworld (that targets > ! > ! the running system). Not true for #1 and #2. > ! > !=20 > ! > ! The dates on these tools being more recent than > ! > ! the files that they were involved in producing > ! > ! leads to rebuilding those files. That in turn > ! > ! leads to other files being rebuilt. > ! > !=20 > ! > ! make with -dM reports the likes of: > ! > !=20 > ! > ! file '. . ./tmp/legacy/usr/sbin/awk' is newer than the = target... > ! > !=20 > ! > ! explicitly as it goes. As I remember tmp/legacy/usr/sbin/ > ! > ! was always part of the path for what I found. > ! >=20 > ! > Mark, thanks a lot for the proper input at the right time! > ! >=20 > ! > This put me on the right track and I mananged to analyze and > ! > understand what is actually happening. > ! >=20 > ! > It looks like my issue does resolve itself somehow, and things > ! > start to behave as expected again after four builds. > !=20 > ! Intersting. > !=20 > ! > ! I did not do the analysis of how (e.g.) tmp/legacy/usr/sbin/awk > ! > ! ended up being newer than such a target and, so, causing a > ! > ! rebuild of that target. I was going the direction: that > ! > ! it is newer really is unlikely to justify the rebuild for > ! > ! the target(s) in question. The other direction about how > ! > ! it got to be newer is also relevant. > ! >=20 > ! > I have now analyzed some parts of it. META_MODE typically finds = some > ! > build-tools to rebuild, but then if the result is not different > ! > from what was there before, then "install" will not copy it to the > ! > bin-dir, and so the avalanche gets usually avoided. > ! >=20 > !=20 > ! The implication is that "install -C" is in use, quoting the > ! man page: > !=20 > ! -C Copy the file. If the target file already exists and = the files > ! are the same, then do not change the modification time = of the > ! target. If the target's file flags and mode need not = to be > ! changed, the target's inode change time is also = unchanged. > !=20 > ! -c Copy the file. This is actually the default. The -c = option is > ! only included for backwards compatibility. > !=20 > ! -C might have more of an effect in a reproducible-build > ! style build process than on a non-reproducible-build > ! style one. >=20 > Yepp. "install -p" is used, see /usr/src/tools/install.sh >=20 The code for the _bootstap_tools_links uses "cp -pf", not install, to establish part of . . ./tmp/legacy/bin/ . (Note: . . ./tmp/legacy/sbin -> ../bin so is a via a symbolic link.) Before the "cp -pf" there is a "rm -f" deleting the target file before the copy: the prior file in . . ./tmp/legacy/bin/ is never directly preserved. (The new copy might still be identical to the old one: the source path one might happen to be identical as well.) # Link the tools that we need for building but don't need to bootstrap = because # the host version is known to be compatible into ${WORLDTMP}/legacy # We do this before building any of the bootstrap tools in case they = depend on # the presence of any of the links (e.g. as m4/lex/awk) ${_bt}-links: .PHONY .for _tool in ${_bootstrap_tools_links} ${_bt}-link-${_tool}: .PHONY @rm -f "${WORLDTMP}/legacy/bin/${_tool}"; \ source_path=3D`which ${_tool}`; \ if [ ! -e "$${source_path}" ] ; then \ echo "Cannot find host tool '${_tool}'"; false; \ fi; \ cp -pf "$${source_path}" "${WORLDTMP}/legacy/bin/${_tool}" ${_bt}-links: ${_bt}-link-${_tool} .endfor Note: This is for the !defined(BOOTSTRAP_ALL_TOOLS) case. Note: the code uses the abbreviation: _bt=3D _bootstrap-tools _bootstrap_tools_links is built mostly in terms of _basic_bootstrap_tools and _basic_bootstrap_tools_multilink from earlier logic. For reference, showing what ends up handled this way: # grep -r "_bootstrap_tools" /usr/main-src/Makefile* = /usr/main-src/share/ | more /usr/main-src/Makefile.inc1:# _bootstrap_tools_links variable. /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3Dm4 lex /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3Dmtree /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3Dcat /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3Dcrunchide /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3Dcrunchgen /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3Dmkimg /usr/main-src/Makefile.inc1:_kerberos5_bootstrap_tools=3D \ /usr/main-src/Makefile.inc1:.ORDER: = ${_kerberos5_bootstrap_tools:C/^/${_bt}-/g} /usr/main-src/Makefile.inc1:.for _tool in ${_kerberos5_bootstrap_tools} /usr/main-src/Makefile.inc1:# The tools listed in _basic_bootstrap_tools = will generally not be /usr/main-src/Makefile.inc1:# case we use the = _basic_bootstrap_tools_multilink variable which is a list of = /usr/main-src/Makefile.inc1:_basic_bootstrap_tools_multilink=3Dusr.bin/gre= p grep,egrep,fgrep /usr/main-src/Makefile.inc1:_basic_bootstrap_tools_multilink+=3Dbin/test = test,[ /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.bin/cut = bin/expr usr.bin/gencat usr.bin/join \ = /usr/main-src/Makefile.inc1:_basic_bootstrap_tools_multilink+=3Dusr.bin/aw= k awk,nawk /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.bin/file2c = /usr/main-src/Makefile.inc1:_basic_bootstrap_tools_multilink+=3Dusr.bin/bi= ntrans uuencode,uudecode /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.bin/xargs /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.bin/cap_mkdb = /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.sbin/services_mk= db usr.sbin/pwd_mkdb /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.bin/ldd /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dbin/chflags /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3Dsysctl = /usr/main-src/Makefile.inc1:_other_bootstrap_tools+=3Dtools/build/cross-bu= ild/fake_chflags /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.bin/mkfifo /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.bin/jot /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dsbin/md5 /usr/main-src/Makefile.inc1:_basic_bootstrap_tools+=3Dusr.sbin/tzsetup = /usr/main-src/Makefile.inc1:_other_bootstrap_tools+=3D${_basic_bootstrap_t= ools} /usr/main-src/Makefile.inc1:.for _subdir _links in = ${_basic_bootstrap_tools_multilink} /usr/main-src/Makefile.inc1:_other_bootstrap_tools+=3D${_subdir} /usr/main-src/Makefile.inc1:_other_bootstrap_tools+=3Dusr.bin/bmake /usr/main-src/Makefile.inc1:_other_bootstrap_tools+=3Dlib/libbz2 /usr/main-src/Makefile.inc1:_other_bootstrap_tools+=3Dlib/libz /usr/main-src/Makefile.inc1:_other_bootstrap_tools+=3Dlib/libcrypt /usr/main-src/Makefile.inc1:# All tools in _basic_bootstrap_tools have = the same name as the subdirectory = /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3D${_basic_bootstrap_t= ools:T} /usr/main-src/Makefile.inc1:.for _subdir _links in = ${_basic_bootstrap_tools_multilink} /usr/main-src/Makefile.inc1:_bootstrap_tools_links+=3D${_links:S/,/ /g} /usr/main-src/Makefile.inc1:.for _tool in ${_bootstrap_tools_links} /usr/main-src/Makefile.inc1: ${_kerberos5_bootstrap_tools} \ /usr/main-src/Makefile.inc1: ${_other_bootstrap_tools} \ What the prior installworld did for the analogous files is a separate issue. =3D=3D=3D Mark Millard marklmi at yahoo.com