From nobody Sun Apr 06 20:27:09 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZW3mV5gFQz5sRsx for ; Sun, 06 Apr 2025 20:27:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZW3mV2xdzz3jsn for ; Sun, 06 Apr 2025 20:27:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=gLXLIBQb; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1743971235; bh=+XLIW2fpCwnwESHdNMkrtP3s559uFe/Cu3c8k37KxZo=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=gLXLIBQb7Qmg2XpKhSsU5kZQxO7Fji7LgZMEJux5zLyXXozFnThnIE1Brhfetbpu4eH/jzlEs/qnLaH/egpmaOKUmOBcWAi096rJUR+pzgaC12qoBkGoqZu9V3bPgzP4af1PMrTZQmkRQRBMBHXqrCev3h2wIL+CuLEWLiWl+7nLFAOv7BeoQKCVwNtNMyf6TRGfCzMpaqwa6gx2SORwpVV6RzhqO3MiI7VBVJuEKok4Ua2L9usXIKI3+U29HmxJ/IHKimINlluHoQtYl8bG+acqZ3EkWazvOApd9o5Jpp7hu9y2WyaLZE0rGRrQdyRMxprVbU3ER4CULSfnhqCzuw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1743971235; bh=Iqoy8UvZMSvhdnWE8KqoL+vjk1ALd9WMd6UfJBDHZGa=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=kt12jUlezQppLS/wfMxK9I1tlmsYv/E6xyzlQzPthrDSm/1KyYIG57ZpDRToh8B0gRJIMYPqZBY7hZ+9MNUkjTAEHCorFE8OgOqOZvxCDQBiPZ6FlVyH/4L5ofB5GBu5TjiSb4IFG5bpTW88VOAc1iknI35vmmv84LzwB1Fa/uT9hk54mKGNDuzXTL4j73LeQHwqXcAm8Rpt/OfRvSD1eRGonnZ5ZB9AtbqqKc2bg3s63ZnfmZNBLUOR0MaZHhbONIbCFbIgC1aKNFJqL+rx5PW4vuy3BYz5T7hX6v/uvi5xKhd9Sjf4BRkztx6KLVXMNeajXLjbd3QnhhQSZwtMBA== X-YMail-OSG: Gs0mdXoVM1mOSPII2jQTzghDVn1sIgGY2hvsSBE7E0rC2jgP6I1QFy7vWKJOKoS .X15DOFTdsWwYKdi7Uld4sYx3TCX5gsYqpu30_hG97QPwVl2vCnU.WHFLi6DiAT49tsg9eewZg3k xGnbe.yVztcpzntwKvIOUsuP9cBCDnq44MMC20ohfgyrg5w6Rq4TrAfK_3wIebt_d38lP3_1TaWB ONeC4F3tLuheGVZzMdlmVjcXW_B8LFaGaNdV8o2Z5p6ibGjLdQl8DjBEmk0yq8dq6v5qIzv.Hl0K owPNGSkoyS52D6hzTPg2ZEb5i81eoz_dECxlVbxqLPikvNEL4v0MCNMVUu4MNrVX4daLz0oh5ueK FLh8SS0U5SVVUFLhsz25z3pbJweZ9lSKADTwyXItjbKpGEQYlITHwC9JWIrwXMjDfWdKepi5dupN C4ZnG5oIq8BJaCAIPKJl91tVcXXTi7zFGjZdhmv6s6zCWI0DpCqzH4c1V9Pj4hJ.40VAxV1KIuSR ss2oxBfaDagLcC7pDkRbZsvoHhHFnPHoUVNAPLnMB4ujz5e8TGp7Dag2b95LdjsL5DjvW104xFJ. LZ.4s2TWx_YxyJYGcSPc4eJxD.ckzgBBiaIQpblBQY0DmcJ7X1VDIsCxv_kMXOqAOZu_PqlGbyQw wMuTn6mppDrwaTvqdAfVG.zg0WG65ZN.QQHXuWPQYSRRnGYKgCJidQv4BSdjaVL5ALAebcKelWaa 9tBaB.ba3uHrD.eWDmw1rg_46TsHkvDPyiEl0UBEh3v0yHIf4YX.QhFOSAhVY9bHbNf0EAeFPdx_ Q5DjjLwJt3XwqQyuowi2cAzRkrPLd.nf46D1OOWhOw5if6EIHBgU6osBCM7oUBbXRC3_QIA0rsvc G1Fl1YJSQPU1Mv.h0g3PCAcwI_o7TZqdq08P8tD0AWw4nVSqaBqtUzHXRO7.FP5VOKc6GqLO_qXe Is1pHbqxrMJ9WXr3SnswvuYiD4tGPJvPOucgcg9JgFCPtnM9jomYuPmDtq0WdYc.K82nYKpdxBDH RHSU0jJCBTx7KLpTShj6mILudeA_cyWH0j47OY7Dyem7M8lp2a4JcvNzsjklPmPlheIoPid4UXl4 3oUoUrQO5Wvp_NL9_enIr6XbkSHA1vrP5m8fDHv.T3Y1PI2UUonplLYVWvZLU7ucIJQf2PVJYDKQ ZrXTIYtoSzyXkpcyT_QZma_AWGqAtUnIqRNO2XIGZz5DydMTrROKjpwUdcRwT6dSAx4H5kQ6QK0h _Fb_J7hlGrnzKq9MfXT0QvofERialtKc3dfanqVtWsTBKOU6mhyfMnfpF7rNkuHWWEUDHNfugZC2 jdEr6TMMi.KUMX8M5.cej6b2FzAdtJayILf7tq4FwOCjEgNMQGXEsr1UYDkh2bIVUF5flqCujkgu 3l4lE2Ua296GoB4GkjI5FezjKFkR8qnbts3_MfEUKM5DgpTG8DdRrApOD7jOr9evQWGFBumU.u5c CLlkDKpBDkux8Ox_UTUnXINEiBwAtYhqkCszrrL74baxeKIe7SjuEu9T6FTnJs9zFEXcQLMGeAWb RdQLa3BxJu6gbEKMAUbZbhH0MVq_C64tWEOotOyIWUGj1bwwcHFIDdvgvtaz0dttG7KguqJzZQOP UzwgVd0OJT4oeJaKh.VjcJaXy82TmloHUgvMmaa7bph6jW3TF_nmxyGD1ZXQN5Tj2sTNBYpOdiXZ gWcphZMV.cqE79zD7Vh1D85ODnNfjNDCvmxgvdO8n3sm8NRUNX4xBrUA9iv9sa_HoJmZFIocOuZb e0wP9cVVh3ZhnWZ.bL09lx3vxZKhMq3xnmgtHPQynKrLgKYzn35ZsfJOp9gEGz5GwyEnCP6cQ3V. Gdk26x8wTSQAhReTaLgr10iel7ORoVyNyBcfml3YfBQ1oBoAWKyRmxlVQbZGZQwFfUewnJAhQQzF RdagoI2M0qy3cUAmezOfUA7LvM4ASR6t7wO79HIEHRZ9tuwjq74MTnJs_OUSktKjqJ67HQKSSniz nmRvWzNhSzp8GROcuPL8KjWGfq66IJNOV1jFQChuPauTECudgDPozoPTGBufPLEHgV8yb_q7veGk uk.rd8arGyKaY9hDo5BYBsqFngHzTWF5c0YHM_KGnVREbEte3LCO2Ckzt8ugNeHnDJvrXVHrsILj 9knw3LrUgb7fnIVkGrcPlo_3f7jG5AoQgtLWJBybt7wEiruRMBI_hIY.ypxec0ihYwMsDCoTy.SP hDChP1vsMnrW9WiA._HIBqYXp5CUBYMpm3FCtG9u_TutpkTGh4gJ2kiebHW0VUfXchfiLlm.hxEh 3 X-Sonic-MF: X-Sonic-ID: ac6dba3b-721b-42a9-80a0-e77ee9b3ffd7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Sun, 6 Apr 2025 20:27:15 +0000 Received: by hermes--production-gq1-5c477bf655-s8xr7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 5c87b60c86fc767433d06885e9e208df; Sun, 06 Apr 2025 20:27:11 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Re: UPDATE: pkg 2.1.0 looks to be making official bulk builds of packages take much longer From: Mark Millard In-Reply-To: <84FBBAF8-025E-4B9D-9797-51735567A8DB@yahoo.com> Date: Sun, 6 Apr 2025 13:27:09 -0700 Cc: FreeBSD Current , FreeBSD Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: References: <8E2FBAD3-EF6F-4D99-A340-21F8FD19AE0F@yahoo.com> <84FBBAF8-025E-4B9D-9797-51735567A8DB@yahoo.com> To: Baptiste Daroussin X-Mailer: Apple Mail (2.3826.500.181.1.5) X-Spamd-Result: default: False [-4.49 / 15.00]; RBL_SENDERSCORE_REPUT_9(-1.00)[98.137.64.206:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.995]; NEURAL_HAM_MEDIUM(-0.99)[-0.994]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; DKIM_TRACE(0.00)[yahoo.com:+]; ARC_NA(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from] X-Rspamd-Queue-Id: 4ZW3mV2xdzz3jsn X-Spamd-Bar: ---- [I thought of another question.] On Apr 6, 2025, at 10:53, Mark Millard wrote: > On Apr 6, 2025, at 08:39, Baptiste Daroussin wrote: >=20 >> Le 5 avril 2025 06:58:12 GMT+02:00, Mark Millard = a =C3=A9crit : >>> [This is an update of earlier notes, now that I've noticed what >>> is different for the contexts that not seeing larger time frames >>> in the Mar-29/Apr-01 bulk build starts of official package builds.] >>>=20 >>> The context here is the official bulk builds started >>> 2025-Mar-29 (beefy 17 and beefy18) or 2025-Apr-01 . >>>=20 >>> pkg 1.21.3 was in use on beefy 19 and beefy20. These did not get the >>> significantly longer time frames. >>>=20 >>> pkg 2.1.0 was(/is) in use on: >>> (beefy17 and beefy18 are significantly slower hardware than beefy21 >>> and beefy22) >>>=20 >>> beefy17 main-i386 168:11:08 vs. prior maximum 74:19:29 >>> beefy18 main-amd64 168:11:10+ (so far) vs. prior maximum 96:28:10 >>> (beefy18 still has 9338 Remaining and still has status = parallel_build:) >>>=20 >>> beefy21 142i386 50:40:16 vs. prior maximum 40:22:44 [141i386] >>> beefy22 142amd64 58:51:19 vs. prior maximum 49:37:29 [141amd64] >>>=20 >>> Note that beefy17 took around 94 hrs longer, more than doubling the = time. >>>=20 >>> ampere2 main-arm64 is somewhat over 1/3 of the way along but also = looks >>> to likely have a much longer overall time frame. >>>=20 >>>=20 >>> Note that beefy17 and beefy18 had/has: >>> (has large time changes) >>>=20 >>> Host OSVERSION: 1500028 >>> Jail OSVERSION: 1500035 >>>=20 >>> beefy19, beefy20, beefy21, and beefy22 had: >>> (mix of little vs. large time changes) >>>=20 >>> Host OSVERSION: 1500035 >>> Jail OSVERSION: 1402000 >>>=20 >>> ampere2's main-arm64 has: >>> (has large time changes) >>>=20 >>> Host OSVERSION: 1500035 >>> Jail OSVERSION: 1500035 >>>=20 >>> So: The new Host 1500035 use is not the cause. Nor is Jail 1402000 . >>>=20 >>> =3D=3D=3D >>> Mark Millard >>> marklmi at yahoo.com >>>=20 >>=20 >>=20 >> yes this is known and expected, because ofnthe support of = provide/requires in a way the ports tree can use it (pkg add) we added = some code which introduce performance penalty, we expect to be able to = improve performance again by using those features in the ports tree for = real (right now the work is in poudriere, then the features will be = added to the ports tree) >=20 > Good to know. You might want to send out a HEADS UP style notice for = folks > that build their own packages. A lot of these folks do so in order to > get security updates quicker as far as I can tell. A known large = change to > their build time frames could be important to their planning. >=20 > It might be appropriate to suggest temporarily manually controlling = the > version of ports-mgmt/pkg used to be 2.0.6 (or before) for those for = which > time to build is the most important in the interim. >=20 > It looks like ports-mgmt/poudreire* would not need to be controlled = (yet?): >=20 > It looks like ports-mgmt/poudreire has not been updated since = 2024-08-25 . > It looks like ports-mgmt/poudreire-devel has not been updated since = 2025-02-09 . >=20 >=20 > The notes may be of use to some others. For me, the likely implication > is to stop updating my ports tree builds except on the fastest amd64 > system and fastest aarch64 system and to stop building for armv7 for > now. (The fastest aarch64 system does not support AArch32/armv7 > and the fastest that does support such takes over 5 times as long > compared to fastest aarch64 system.) >=20 >=20 > Going in another direction of questions for folks that do their own > bulk builds of packages: >=20 >=20 > Context for the next paragraph: Already "using those features in the > ports tree for real" for someone doing their own package builds: >=20 > Will bulk -c builds (for example) always suffer the longer build > times by not having prior information for reference (because of > the -c)? What sorts of activities would destroy the information > for the next build attempt if that is an issue, making the next > build take the new, much-longer overall build time? For example, > updating the poudriere jail's world? >=20 >=20 > Context for the next paragraph: Just before "using those features in > the ports tree for real" for someone doing their own package builds: >=20 > How will one get to the point of "using those features in the ports > tree for real"? How will a self-builder of packages get to the point > of being able to test the build times in the "for real" context as > well? >=20 >=20 >> bapt >=20 >=20 > Just for reference for official main-arm64-default bulk -c -a (from > scratch) builds: >=20 >=20 > p25bf3a3260c7_s680d34896c3 queued 36447 > and has built 15523 and has 19479 remaining, 134:23:16 so far > (will have built up to 15523+19479 =3D=3D 35002 when done, if it = finishes) >=20 > So: 12 to 13 days (around 300 hrs) as an estimate. >=20 >=20 > The prior longest main-arm64-default official build that completed: > p02dd5021d6f9_s717adecbbb5 queued 36466 > and had built 34853 and had 0 remaining, 163:20:35 >=20 > So: 6.8 days or so. >=20 > Overall: very roughly doubling the overall time when the "for > real" context does not apply. Additional question . . . Which of the following are the stage(s) that take the extra time for an active builder? : check-sanity pkg-depends fetch-depends fetch checksum extract-depends extract patch-depends patch build-depends lib-depends configure build run-depends stage package =3D=3D=3D Mark Millard marklmi at yahoo.com