From nobody Thu Feb 23 05:09:54 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PMh0y14Qzz3sywX for ; Thu, 23 Feb 2023 05:10:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-22.consmr.mail.gq1.yahoo.com (sonic309-22.consmr.mail.gq1.yahoo.com [98.137.65.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PMh0w1XYHz4Y7S for ; Thu, 23 Feb 2023 05:10:08 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=SonYrXoK; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677129005; bh=33+qL4ysoOv5xjzOEquo+PuFD2l6lAW5DeaWbnf0YSc=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=SonYrXoKnTNYnNEryFuOdKoMPmok/BErTHQ+Yov99Jtzahhc4giYNMYrX9qaHquTtg9CWIDjsec9w42SoXYcHT6Vg9+RCrdtwbvLTnnjFVhw2bdjNtTBrm2sBhxhUUEdQPLO2fuhWstGK/T1+we+B8ezUo+SvN/T02F+W3wBmP9uPxsW0uXn1hHsUPCai166151tQyssVTy8s8lys4d4O0l2rh8XkynsAO+9430iE0LDeuMVTqheD4Ead7Stm7+rqVBESh7ElCfU7bUUZYRsm1cgsF+yACGccBoPKygl9IbAOcgJrapVUqBQHy05QZL4DHIw0aWmb1VvrDsLYJctmw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677129005; bh=rOsbY5aeuiCMk3AJ0Co6jLVtPMZ5xARGcB++shROzcC=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=CW0BOixa+7NqA9YJ25IzEQg0DH6BP9ilqNb5x2Q3Rvbu9hLBjAi8eUkJW2su6BcGaxWEqh74I2ny8uZtur4hLZmsw4LWhltQmmYJNczMq5ilBHy8qUSNYI8W1G5PsF/6itqKoCy23Z76NR1mx7yMgw/FGXqT2NPJz32z4uWEqZ3erolM8TMQJlGJeFSa5/I3WfsgdbjlkklkKLpcJS1PK8fjOE1j9zcjhwmYRRQgp6CY7r5Eb2UDksn505sZI1WAGVg+BMDUzLHPHY6BGZeVH48ptk4napCvkTIl/gaDH6wyn2+Wkh8pnIQcHyI0ez4QYs7TfSLoPBvN1v3/Rw4Pog== X-YMail-OSG: qu_ILrsVM1ki7uhUWojsTsoApFR.1kUpzEwEupQAeA9KXIr72KLennCGBPuF0CC KlJXB6tAtDYLgWmv6wIVGjaChGVXOtbZUZXwuEmh_U62sr0cHn0iUp6OR.nz5bq.rH3XJelK9rOG P31EDp_IaW_U33_65AkipXqkj8P9YcH.IRgklTWQ3tER2T8VKIzioGtPgPbZQWUU5juLF9zbfXbo 02dDqglKWgdujMtvd435_ZFbiN.rtkNqm.LqTFtZg1CL2TJSRE3jJN4.P8C13Z7xIowT7XkLGVIE Ln5hD9dNw.rNud9H.FmfFSdd4LtRcxIoGovNT3_VdXQZQIJHY0tfyxZjTemS_1WQHUakCVN8iivN xH5gnit7wYjYacPfAieSuMXPO.44TgKR1YPeOtPhL.UA9M1TNArDlCnyr6uQVrqcf947FzvQm2it jVJ3Kv8..HD30y6LT0gufXa6J3ZlH188ltm7EWqdexKn4fGvbKAsi36B3dIKPM6O4fUSVbFErs49 J8xi.fZKjrmbrbEQ90PG0IweFuCk3kPywDogNCBbIa.flEe.ivULZt3P1Q3Kg5Gma4gwMCQpqWMw EepJobGZuCEdk3.zGwPxYPrATpNArwjicqzkPloWNhFhDTrAJ87C2eoCW6ub755iKqrgO_sAKqB4 KwVWKCUnqaGfC_A1EE6kDsfwEBgqEy2EiSDihSAciHXiZWdKOB2y8IpOYNiD_9nCgZfP32hKeWj5 y6kilOrmB.KYCFDoRadbGyA275arfM4dIJHNO17fieOTye_fTvyDWsNSGpJEvMVxVaTHq_uoljBj 1HB02dEd7Cf7S9pXED1Z0I3bgHKGh0rVgaAkmasXPNPvnchKNLQPUcptRsokZrxmmSLeFZXCgN_a 6D0wj5hqArAF_XdFHu_IeX2w1eycQgNteUYtaf6ZDa0KCuXUQnHpsYY8aHnzIKRVqH2.h5mor1N_ 4EeP.g34FSb2H5S6XAgKbjFh_MAy_YZBR4WjBB4TUM5ICAuLwOGN1B_l4iNeMFxnJmOMF30ncc0_ qb.BR2qwtuILmFriQeHAtTpoydS5zcI.By2SVPtzG_J9bcDnxM08GLRjkedJyFDO_Yz2kXm94ZTo dyfazSRzAgmjDfwKkeTHrgnndXPDOamyviU4IveomTz6vdlaI_V0vLPIw3SBLS97ziWHX2KGWLJb UriyiTDzN7DPnfeVLX03WNo8Kec_So5do0jTL_jfu1Hg5hBMeYQ84CFEPu0bqM.hfWTF2zhar67d rO4SfaP5pNfSnboTBBj023p6HIerxGXfHES9QsWsEvjBfny0Ldazeekeg.ajc_Qq491lW4.obVg2 DDFyj1KZs1lv1_PMNRw.L2od7h2.XO4ATaWP.9mX7Eq.0_19IxrVLLehdtRUE2T56Fa6FXrHfVQU syGldFQiTE7O3VIo44NNTOxyT4Fj_0a6CerilOlp8cOtDJihPVtVBRUKPsnqvp2iaKZAY4c5hlfy N7buPZJTcd7c96c7EG6PKZufwsEpoGS2xnQSOPivJ8BrDLJUGoiREouRSf.lK9CTJgficOXSedWH EoIb1bT6PU.NgrqvFZSrQLN_X7buzMRQG_IKmE5WQk7G040_rd3IDt_4jZQAEBADIz2w_flj7mZR 0zma3c5OhIc6or4lL4ENI5Ko.vkYprMvKxjhDoByTWHyNwNT8OaHGcDXev9RwfpYHcNJUTENk0ou lPpgHu4x3kE4BAUUS_jRvatp5b9P0ZVhbeKllxqRGGywiBC.5lH03Shm0WGXoM4Z03sJ7N6gHSHr L71ZD9puRRCWlQvEHe54SvD5iSchPY4QQohB7IoqaaQ_9ePO09Zxv07JTkdOS3je5kfO1Eo4C_1w UlbqFzbcDaslQldMXK29McE4mJhnYxxLq6GstJzOST0i1pO.BENOWkYzMZJF_5ZbPap1VSIms_Hd tCyx1QQQQ7qfN9SQeNru_4rGLoPz0bFwkn0A7ptDUgZkIIbmPS_KWb8LcxIwOxOKUiwmgRYPyIYN USjJtCZbr9eUrSH6LowK8K_GhFLBBYKCaHB81AVaQzb39Ut8mmxPP60BSJHRG8oi09JYVBo9dVqI w_aga0A4vGy0anac1XTJSQanqm5FrO2ZztAdd1QsEo3vx58mmIzuC9KGUS9Hz_ye4mPIYNNUbQ6p Rj6pjlJT.S_a4Yo41bY.I3_DGTCClbLuDoA.RZqaqAqUA9TOohaVFcNGE_IyDkzd6IagPPZkinzj Dn9FMSNdni10ApyGo4VCpMsNs232Mw41RjhPRO3JN89I8v.XAvWwn8H0elT5gW.tWyvLhaFlVuQ- - X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Thu, 23 Feb 2023 05:10:05 +0000 Received: by hermes--production-gq1-655ddccc9-59dl9 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a4e72563c9e1465c0fa374d16a1c3649; Thu, 23 Feb 2023 05:10:05 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) From: Mark Millard In-Reply-To: <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> Date: Wed, 22 Feb 2023 21:09:54 -0800 Cc: Bryan Drewery , Current FreeBSD , Peter Content-Transfer-Encoding: quoted-printable Message-Id: <7FB6F619-6E71-4075-8A6C-573564371DD5@yahoo.com> References: <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <73088.1611797582@kaos.jnpr.net> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.990]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.148:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@FreeBSD.org]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.148:from] X-Rspamd-Queue-Id: 4PMh0w1XYHz4Y7S X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Feb 22, 2023, at 19:47, Mark Millard wrote: > On Feb 22, 2023, at 17:18, Simon J. Gerraty wrote: >=20 >> Mark Millard wrote: >>=20 >>> Thanks for the information. >>>=20 >>>> strings `which bmake` | grep META.IGNORE >>>> .MAKE.META.IGNORE_PATHS >>>> .MAKE.META.IGNORE_PATTERNS >>>> ${.MAKE.META.IGNORE_PATHS:O:u:tA} >>>=20 >>> The -dM output's "is newer than the target" lines >>> show the path from before the above transformation. >>> (The :tA results possibly could use another >>> sort/uniq sequence for the realpath results?) >>=20 >> That indicates the above IGNOREs are not working. >>=20 >>> I've been pondering things because, so far, my >>> attempts to experiment with this has failed to make >>> the -dM output lines for the paths go away and it >>> still does the related build activity. I've been >>> trying the likes of: >>>=20 >>> .for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide = dd egrep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp = mv patch realpath rm sed sh touch truncate uudecode uuencode >>> xargs >>=20 >> Is there anything under ${OBJTOP}/tmp that you don't want to ignore? >=20 > More than just _bootstrap_tools_links entries end up in > ${WORLDTMP}/legacy/bin/ (so in ${WORLDTMP}/legacy/sbin/ > via the symbolic link pointing to ${WORLDTMP}/legacy/bin/ ). > So: yes. >=20 > Also, OBJTOP is not constant over all the parts of > buildworld buildkernel . Having the late-substitution > form of notation ${OBJTOP} might not be appropriate > for the content of .MAKE.META.IGNORE_PATHS . >=20 > I'm trying to figure out if there is a stable way of > getting a path that would not suffer variability > via late substitution.=20 >=20 >> Otherwise you could simply use >>=20 >> .MAKE.META.IGNORE_PATHS+=3D ${OBJTOP}/tmp/ >=20 > (Ignoring the variability of OBJTOP issue . . .) >=20 > I do not expect that would work: ignoring things > it likely should not. >=20 > Also, I'd rather grow a smaller set of ignores > gradually to make it easier to detect if an > addition starts causing a problem and can be > backed out. Starting with everything ignored > would make things much harder to figure out > when ignoring creates a problem. >=20 >> You might need ${OBJTOP:tA}/tmp/ >> or both. >>=20 >>> .MAKE.META.IGNORE_PATHS+=3D = ${OBJTOP}/tmp/legacy/usr/sbin/${ignore_legacy_tool} >>> .endfor >>> .for ignore_other_tool in ctfconvert objcopy nm >>> .MAKE.META.IGNORE_PATHS+=3D = ${OBJTOP}/tmp/usr/bin/${ignore_other_tool} >>> .endfor >>>=20 >>> in what I use for make.conf via: >>>=20 >>> __MAKE_CONF=3D/usr/home/root/src.configs/make.conf >>>=20 >>> It is using paths that match the -dM output lines ( sbin >>> use despite sbin -> ../bin being a symbolic link). >>>=20 >>> Note: WORLDTMP is not defined that early, thus the ${OBJTOP}/tmp >>> use. >>>=20 >>> -V.MAKE.META.IGNORE_PATHS is showing the paths I would >>> expect, matching the -dM lines. >>=20 >> Do you have example? >> I really need to add some unit-tests for these... >=20 > You may want to wait while I see if I can come up with > a better example context to show things. I only just > noticed the late-substitution potential issue and > started looking for a way to avoid it, for example. > (Either a value that does not vary or a form of > causing up-front substitutions in my make.conf .) >=20 I got some useful information about one type of context that is odd and creates a far mount of "is newer than the target" notices. This is an example of something that always "rebuilds" via a . . ./sbin/realpath "is newer": = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/aac/machine.meta: 23: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/realpath' is newer than the target... Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/aac/machine It turns out that the .meta file is for a symbolic link: # ls -Tld = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/aac/machine lrwxr-xr-x 1 root wheel 31 Feb 22 20:19:27 2023 = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/aac/machine -> = /usr/main-src/sys/amd64/include . . ./sys/modules/*/machine being a symbolic link to a directory is = normal. It turns out that . . ./sbin/ln "is newer" examples are tied to a similar issue: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw= k/awkgram.tab.h.meta: 12: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/ln' is newer than the target... Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw= k/awkgram.tab.h But . . . # ls -Tld = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw= k/awkgram.tab.h lrwxr-xr-x 1 root wheel 9 Feb 22 20:18:24 2023 = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw= k/awkgram.tab.h -> awkgram.h The .meta file is for a symbolic link, to a file for this type of context, not to a directory. (All the examples happen for what I'm looking at happen to be symbolic links to header files.) It appears that for symbolic links being the target, META_MODE does not respect .MAKE.META.IGNORE_PATHS . =3D=3D=3D Mark Millard marklmi at yahoo.com