From nobody Sun Apr 06 17:53:49 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZW0Mk6Df7z5sFJL for ; Sun, 06 Apr 2025 17:54:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-54.consmr.mail.gq1.yahoo.com (sonic316-54.consmr.mail.gq1.yahoo.com [98.137.69.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZW0Mk55rCz3ByD for ; Sun, 06 Apr 2025 17:54:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1743962044; bh=rSKABcAMzudbZ9nTrBEKdol17O8kjYlDYfZyEtxUJPQ=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=HaZszwhlTXi0r/fgFcUVk89tY3K4ckHb0pAHj9/kAFv6ItYjfmpI4ZDQuJFYUorjpr8cR93cVx97UWtBWCPlj9j9wHZyTQWhnmYXtBblQnJUgSMCc/qfcDEEBVFZJuQ0OzzY9cMKnBsx3IphLTnh8G4x9fv35kWuMp/IKnmd7WwHjLO4koboVb5S6lrRNOG985Js3e/Mk5YAB6kfcZzhvuEI6YVlmr/3qJHh7tk2Z22bRGzX6erkgH+yFjgW5WgMY3lfeIzVDWKo/S/IYDp+yA28xwH+74xyDHqLcZ4ARJY69VyEph0zXzMcba+Ehqf63VTmpvxFUaJyYlw/DXVgHA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1743962044; bh=g5eWcEyaUQ7366KxNkjqCnUK7OoP1P5AoXH1JPpLB9h=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=uR5/bGTRf62JyG+uOCdFaQZWkhV0u67suXNNndczhQ2H6tNfBsIhL3un/8USw7AebgGsj3pgDmuvBQOjMiFyd0pGsN5dWRIpgBjaltCGxqDedVTYv3xWiIF05LSC6cJ44ujTPu6uwfgKQLmgO8tjlfwMclXLOpxSIMvWOalrnK+Cq03p6Go3Z6d7jStJ66w9iWdr/j3snJJGT1JopcOOh0wIyyLi6QpUZytwWwuem+7ccV7fs+qbTiNxBr2oL/Zi7Jn5sLKZ95AapYRYvsmaxVwriETWUXFfraK8rdMo9kWIYTvvHcAHi64uwtkCjDjZnV954egU2UUsz24/ncCpqw== X-YMail-OSG: _0SxqS0VM1lt6BVJRhK0_eGVAw.kJON1lfSA8dbWX2aufRi.jdfUoR0sL83obR9 WOTTk8PeSoe7g8TooaYeoArKUjoL0dtuAfsz4lLyRWmnqQ1BIFGG5UyTipRHwn32VpKXUzBpAer4 QbTEagoJvU6fSQkeOWuDm9V0z2Jo2vma0zihoZxHbOpf0B5FC0G.PgUbuRY_b2rmrL15_rk9_viB KJKSA8Z8MDJNQp4.5YKhekIPpvAb_9.oTJaUqNsmEFw25RX6GH9WovoPN0q6bFvhTSzTjoE2j9iS xvWSTMumG_1XO7CYcDCbLtXUmTLCCRcE.luEbXjtAdfCQbL.i2XtioQcd2BRfLV_AGwVqyi7WH6_ Pe4vPin12VkzojkQQUdrye_LNWspV3x_85fjbn.FfLd70TqLxTFRg4QmKyVBfZ6N7KpxhDSmAtgO E1vVXivBYH0y6lQqt_t.6JAnQDpqhmMz4fxTHTn2TUmyel8TEFoDndy4rDLInoenqhogk6UroULc v3VWKWGd7sH4un2TdAsAtQMt8Gw2ZSGZzsG.kH3aAflTvZXTldVd.Wns_lAefrNtDl9rJF6l9fYQ CygtCGKwKShmXJ.CYWOzQPA8x6fFpuBX5RUClWlGZHaTkbkU4_OypZ_CHmLvbuHqFDGWYiNVlqi6 YK9SCSLpPB7GHGbGrk._DF8MnZYPStnQjP4SM4HKcGRDFGnzz4HdpU0srrSgdHa_uv.f2_V2yZgG WN1HQPXTMSUjAzs9Dg5WdTzNCuPlMu2qvw1nwR8spfiZ.by74q6EbFVC8gved9P70sOSIGbdNU5W wbV5T.lumuu7O5QX1TLv0sNr_0eYjf0R4D0Vtie.wqPd.k.DTHSqGjx6twT9jZ.ZcEeMnTP3499x PUP5EyfkRUDQFo9YaPL8wE78TkIMmeaVw_mz6isg_K4bo6IVpGOBQVBLBS0G0Gh3JPpNhUzdSGNd q9qDR0UeAjL95yBpBFHakZrasQLc67yCL6UwQRnjWYqw3tXTnfKk55GoVgI8wBtlj0tjyLfFZx2a H05v4Mmaa27.fNX.pB9E4cmR_k_id5pKAXIrrdLc358eSpm2kAhBKOkzd8DlgA0gpD9bMmNueYj_ fLa2NEeeYXyb1SCIMzurG6_ZlBwgKmmKzRbgVlA6A.yf4PwvAoRUsOhCqOiYlX4GqyDDWDxyoN0H __MbB5bSnU2HJxmUy1eunY00OvKYJrU7n0Q9wzmNWKfajYNYkHw7sJrLMxhcqzkh4eCheTpp.FB4 clDBJHSBhwderry33gqXWq4sCuXawc9j3aJEI7VJs7JHku9ES2XH1TD6_gFciOBajL7Kr3TG5GUC nMOxegzB7PIFJvtFjZb2E99D7f93NVl8y9G4RhXJwvootT4Ze6U.w88NjfdWfanoQzrlQRjGDXNk NzHcyxTIOovynWSRVzdodxUli6zcVNrMPViVond2J.e_4NyKv7lU5pQZmcAgbzztkmLaK35apC4g CMTNo7N.ldCbyH1x.34Nh_W18sLIqZi.1KEQKSaB.vLvOx7XfVad8u6RT6InMLF2s8T1RUM2v.p6 pCntC6AKsSDqJhPkBsm1mR7DAiAqjliflmNd_uN8BH1BCXXYmbIWJ.am6vwQ3BDHKNUTJw5gCgs8 WHq.vRjGU2LjRdDsI5UjOT57N42DZN6qFwYg20f981jTp7.yW2yRYCiOxAy9TpstJSs5ge1H.ClJ FAR3fBUrzQFQzDvjyRh2QRZQPqL1QPR8AgMLYau.aevPfm6bDfO3gxgLj9oNRSuTME1hSXLm9QtZ LCY.3sOnuSl1kVrWp5C3uTohJX6TgkvfXZ_k_gFwY6CVylOD.Bp1ymSOJSjtd.Gpga6VHkV2XgpP uNcYhou6KfQtcoXrMS7pojcFBbfbcBxrVXI66yvPj5VnvZIA9RZ2_oaS3gqqSEeSY.8fPEMx0h7p _8820YGhs9mflM2kk01NQJEzEs.7O73KQ6N98c.wmMKq9NgEHHfWaecP8E8AVmJi.3u_ETf9qCLM 3lnXAKn8r_PkVVAl6ilmBoHYRgdmfLKqot9ihaT6xNI1s8aL9jHDQIbwzsY9h312qc1xDcN9InGk K6vHqEt4OFcrkjq1xYQxT5DaBoo3TwGhOXCrmXdrKA9wSmq0pKiyc7erNiEDfwki_3SrlvjMZ7bi 31qpOUFoNlMvUdHDAKu9ImaK_eORBQCqC9RbnSV8IxiCqzhURE9T10TQEcmTtpKQ4.jpyYnW2SJJ 6QpH6bdK7CCkv3HMbGC66W3nzHOORsXgRWE.K8r2KMuExttfQHEdGztksAeXvOi1HMSizCnJQSER VJw-- X-Sonic-MF: X-Sonic-ID: 44cb1506-5724-4ba8-949d-566ea887243f Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sun, 6 Apr 2025 17:54:04 +0000 Received: by hermes--production-gq1-5c477bf655-pmr4h (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 7118208c90ac739641f64070c3bd02d2; Sun, 06 Apr 2025 17:54:00 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Re: UPDATE: pkg 2.1.0 looks to be making official bulk builds of packages take much longer From: Mark Millard In-Reply-To: Date: Sun, 6 Apr 2025 10:53:49 -0700 Cc: FreeBSD Current , FreeBSD Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <84FBBAF8-025E-4B9D-9797-51735567A8DB@yahoo.com> References: <8E2FBAD3-EF6F-4D99-A340-21F8FD19AE0F@yahoo.com> To: Baptiste Daroussin X-Mailer: Apple Mail (2.3826.500.181.1.5) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4ZW0Mk55rCz3ByD X-Spamd-Bar: ---- On Apr 6, 2025, at 08:39, Baptiste Daroussin wrote: > Le 5 avril 2025 06:58:12 GMT+02:00, Mark Millard a = =C3=A9crit : >> [This is an update of earlier notes, now that I've noticed what >> is different for the contexts that not seeing larger time frames >> in the Mar-29/Apr-01 bulk build starts of official package builds.] >>=20 >> The context here is the official bulk builds started >> 2025-Mar-29 (beefy 17 and beefy18) or 2025-Apr-01 . >>=20 >> pkg 1.21.3 was in use on beefy 19 and beefy20. These did not get the >> significantly longer time frames. >>=20 >> pkg 2.1.0 was(/is) in use on: >> (beefy17 and beefy18 are significantly slower hardware than beefy21 >> and beefy22) >>=20 >> beefy17 main-i386 168:11:08 vs. prior maximum 74:19:29 >> beefy18 main-amd64 168:11:10+ (so far) vs. prior maximum 96:28:10 >> (beefy18 still has 9338 Remaining and still has status = parallel_build:) >>=20 >> beefy21 142i386 50:40:16 vs. prior maximum 40:22:44 [141i386] >> beefy22 142amd64 58:51:19 vs. prior maximum 49:37:29 [141amd64] >>=20 >> Note that beefy17 took around 94 hrs longer, more than doubling the = time. >>=20 >> ampere2 main-arm64 is somewhat over 1/3 of the way along but also = looks >> to likely have a much longer overall time frame. >>=20 >>=20 >> Note that beefy17 and beefy18 had/has: >> (has large time changes) >>=20 >> Host OSVERSION: 1500028 >> Jail OSVERSION: 1500035 >>=20 >> beefy19, beefy20, beefy21, and beefy22 had: >> (mix of little vs. large time changes) >>=20 >> Host OSVERSION: 1500035 >> Jail OSVERSION: 1402000 >>=20 >> ampere2's main-arm64 has: >> (has large time changes) >>=20 >> Host OSVERSION: 1500035 >> Jail OSVERSION: 1500035 >>=20 >> So: The new Host 1500035 use is not the cause. Nor is Jail 1402000 . >>=20 >> =3D=3D=3D >> Mark Millard >> marklmi at yahoo.com >>=20 >=20 >=20 > yes this is known and expected, because ofnthe support of = provide/requires in a way the ports tree can use it (pkg add) we added = some code which introduce performance penalty, we expect to be able to = improve performance again by using those features in the ports tree for = real (right now the work is in poudriere, then the features will be = added to the ports tree) Good to know. You might want to send out a HEADS UP style notice for = folks that build their own packages. A lot of these folks do so in order to get security updates quicker as far as I can tell. A known large change = to their build time frames could be important to their planning. It might be appropriate to suggest temporarily manually controlling the version of ports-mgmt/pkg used to be 2.0.6 (or before) for those for = which time to build is the most important in the interim. It looks like ports-mgmt/poudreire* would not need to be controlled = (yet?): It looks like ports-mgmt/poudreire has not been updated since = 2024-08-25 . It looks like ports-mgmt/poudreire-devel has not been updated since = 2025-02-09 . The notes may be of use to some others. For me, the likely implication is to stop updating my ports tree builds except on the fastest amd64 system and fastest aarch64 system and to stop building for armv7 for now. (The fastest aarch64 system does not support AArch32/armv7 and the fastest that does support such takes over 5 times as long compared to fastest aarch64 system.) Going in another direction of questions for folks that do their own bulk builds of packages: Context for the next paragraph: Already "using those features in the ports tree for real" for someone doing their own package builds: Will bulk -c builds (for example) always suffer the longer build times by not having prior information for reference (because of the -c)? What sorts of activities would destroy the information for the next build attempt if that is an issue, making the next build take the new, much-longer overall build time? For example, updating the poudriere jail's world? Context for the next paragraph: Just before "using those features in the ports tree for real" for someone doing their own package builds: How will one get to the point of "using those features in the ports tree for real"? How will a self-builder of packages get to the point of being able to test the build times in the "for real" context as well? > bapt Just for reference for official main-arm64-default bulk -c -a (from scratch) builds: p25bf3a3260c7_s680d34896c3 queued 36447 and has built 15523 and has 19479 remaining, 134:23:16 so far (will have built up to 15523+19479 =3D=3D 35002 when done, if it = finishes) So: 12 to 13 days (around 300 hrs) as an estimate. The prior longest main-arm64-default official build that completed: p02dd5021d6f9_s717adecbbb5 queued 36466 and had built 34853 and had 0 remaining, 163:20:35 So: 6.8 days or so. Overall: very roughly doubling the overall time when the "for real" context does not apply. =3D=3D=3D Mark Millard marklmi at yahoo.com