From nobody Mon May 05 15:07:26 2025 X-Original-To: freebsd-pkg@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZrlJG6JZPz5vdQY for ; Mon, 05 May 2025 15:07:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZrlJF0ySbz3n6V for ; Mon, 05 May 2025 15:07:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=U2rW+H2V; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1746457659; bh=pHB+A88ulO1Zel4kAnYc4gHI1jMUo3Kw+8HceHbWY58=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=U2rW+H2Vd+6h1qH2gIJDtWqLTz8qQTIVecJ3JufZV5dDovDHGTcptTDyQgC1dRYTPI4AYzenF3JL1X64sdDJkKFsCBVHJdVGPsaqwzj6haamnHLEjtFVaikqhwipJ8+gi3AtYbCivlX9zcp7WceN9KFHeym7zk+xN3Whuseofk5iZCqzDTXgJvmTIg/u9JJ6nHuFNNL2RIQG8RnEXBST8QqdMd9PSG6PTy2zAKNFdPw3ytLhxPlFBcsBJSCXNhZr8IxyW2covweeLKsh7I1M68VP9l0G8x5TAf3est0BDkSBi0/MlpDoN4x6CHnZC1uNQqSjEVjX0cRiu7Z/v2aDNQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1746457659; bh=Es8a3xA8fk7PRIzO3N5MIVYGGLdeU8GFjP0G8Oh/8eZ=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=NNHt5RtC2+Rb9Rtbu/tMnlL/ksAU14JrHg4/qzWckV3Um3QvvztFlTqbjyU/uv0hQmY4crrR8FvBWWEa2ZEsxDw+lzypjYlbL4yX5QVGEwvBGkhHSkDwi+GO++YyAln27hrF5qKE75WSNCUDA52c4l8XIJ73Clb59QkgemLR0E63/wJON1xXq9IUJFoX3abeQY7bfHV3ketjcTUYqv9o7v80sP191FcHFKL5yFvKFTW0qBycf8gwD6D2eavM7xBcMScj9A7/0DJHbJFPqxK8fUCT/jdrhfNzTmyjrlNL6gDBdwcnUuHJ/BfIN9eQJyxX9Vl46m4yM9WvhS/PMIEs/A== X-YMail-OSG: TArMg0QVM1l43uZd3f1_mJM3xb9J_.moI4OtV3Rl1z.gtWmesgybK2e7LJNF_O4 kLIQRwtZrtUIZVeMSZvBOJAZRxnYycBWmY_4pwZdS.fhB_3Vj3NK3jiDtLAlrMVAfyfex6dCVSTD EoMZZtFdMlTFHwygzHVRW6VYP_0Elj9DZlD3gBjbShdhydEQrvJVy2J6vj9k7SQptrvq5GVSGNYd QlTNecDxpPyts6HStvfWqaYNPRA4xlZCJCllZfLNOegXyuVgMtufJz.tu0gA4iOFkiV0ELLV.oPJ PptCVWaVsc7vk3r.E1CuigbKOzncXUox6kpgmpLSnalWqg7QdomnovFi__gFWgXfdUz2X8CBSxzI _ly3N8Ii0by0.szUlkxfMMcQO9cqarIYMCez5PudEWwuf17FFOTHV0pF6MeztcNjsU.ZF7uLfddl T9nvxvqa3rXqrm1vK619rj5MonHiTF3Z9smCvYe3jpX6gsGRVEOwaUOfFpVJvkn_7uxIc7v5S07z nuK47nhAfcrhpeUO0g7Wo0qve.Al1nqAH1Sd_GycOd2cJwOvTeZEOLXBze_IJ_XefND1CQHvhRjm eqqKYZUaCYOvyvjjgJK32KK06V4y.pQi.H5xaRzG3H5izJoK6B.I.rYrKzdmqQXAEoXHu.EiDsxO gktL47k0W1VmSHP5ecPsdBeCvIp8f4vRmiBotAd2RNUsPVdtEGrogmYKJvQ7X7cR0Y61FMuVtVRw W.sVzFcIn9Ni0QySIAxSlJRL5U59P15vblTPKfOsuAXGl9l6xBczsWHNaGYva5K7kgFFYGL7DoCk voM2aYip4p8Vl1Cq.HPfADMngulr9LJfnSkGdfjilzFXcqsM93V_p1tW9K8fxX.jWolY4IrjkPmx XamC2696IkjV5GnCqLMnAwCnCAeWbdkDDgfrYaFlJrl6fUPPdFZn12O2jo0Fayxhup9RZ21iOqbj 2BH8H3wcq1IpXfqdHbUgQD2YBJctLysODb30S.Voh9XDsoRtVGVZGbLJVhgJMnI7WqczUoMer1vM hqjkxNO1nm_TjwJsykcBnqBhuyQg1FXrO22DzXdf7AkezCuvWSl1seRIJQ5iKZYVc0GEnxaF_i95 wrPPGHE2b1Qk5rTSVy1KVPV7k9GeZHm7tkwJ7a0ie8sdkL7Z.9dMGGI68rrZhJnDjlEZqxWoS49l Yg.4S5HmENf5WhLBIXtrTyfZMto4az.njWvGATZ2LWn2Snwslm7I6ysvRTyZHycKzxvsfuQRMj5M vOsH2WpZEFfZ9j1mVXcHpUOm0uQyQ7W1Yg6CyholNKpgRTHg.gqdhP5LlTP1IBB29XfX.A58xcaU yDNo5acctgQdQ2GisWccTL81G4M.W8kP_cJ0E0513nGa25YhOPGxgM7zUyKluGFx0f6_ZDRTW4Ll BXALsP46u0TpHH4KSOB.wosoqGE19aZAprCIc9kJbCgvJpqVU9.QeBIkgXe7zSu5Xx0A9hh9qaeV Uq8ZVJuNZgk92eH5J11uOuDOyiv7dFULVFKPIv1g.ZArMyKj_Ck1aqTiZi0HSrqUBprRYtP39wcH XNuhusqrweebHVdg9bVctP_60Znpxl.tOCSnQ9QyrwMVj197yXVLgly9RlfngH2J1bY6xpTagmjg CDcTq6dNG9ycGjC8I9gsRVVsP_ANtxYQnA2HT.6tgnTfOiZNGhqj9E0yuBX1ER3mkenSMSDmMQDu EOtsB2ro2ZY8LKTfRKC9RYYoSvHwbePdBe4tDscHtyDOZeNiq56.e7ovZlU_CZDIVfhZENPYP_zL TcAWAVqfCuDu0nCRcckZy8wqsLJYHEMCbGvUHn13WEgOUKpsKXD3NKHwQCvsyg9iGwrdGLHt_zEm FGtyutI4hr54kuBbFDDz4NqJUN4RF4eLSkv.ZEh1WSbRtmCGqX8uQwM6rr_mQmPG1G7WLCC6uqvp 7wJGfpp62H3HkwRlP4eL1n2oeUfA9SsdqfE77EmabhMRL0IVOncvXesQv30M0VIHo6ihxOJM09Lv C6NfbJnNnwHkvHJDnmfjsdBSYUd.c9aLluYswXXh7NZDDnOhOkaL4Muw2BwiLy.eO4gDF9yDSrEM 6uhss2I0Cr2E6XwPOeLVV9djvKoCxHdsa07.uJm9S971Z5jeeEeWiC4zVe0mpVvpP6Vvem8FAK__ AM5v9nnXCA5cqycjQeTm.Unr.pmLQaJxm8u6mVw_Q9NWmzvlHB3Ts8C2FwPhyFSoSQjXBm1Y9QO3 trgWRqA_q.Ta0AlBk.Nxe9.yCTZKtNQl3BTS_K7CTTND3Sz4CLPu9HyXW.uOzwwZ5v7BT0XKGT4Q - X-Sonic-MF: X-Sonic-ID: e9d7ef0d-2822-4ed9-b159-257edcc58f17 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Mon, 5 May 2025 15:07:39 +0000 Received: by hermes--production-gq1-74d64bb7d7-dp9cd (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 7ec93f4f7860cb3cbae6a65429bbe149; Mon, 05 May 2025 15:07:37 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Binary package management and package tools discussion List-Archive: https://lists.freebsd.org/archives/freebsd-pkg List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-pkg@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Will the future of "poudriere bulk -c -a" continue to be worse than linear for time-taken vs. the number of local packages? Message-Id: <72301E26-9618-462D-9C2F-F7CAC48517CA@yahoo.com> Date: Mon, 5 May 2025 08:07:26 -0700 To: Baptiste Daroussin , FreeBSD Toolchain , FreeBSD-pkg@freebsd.org X-Mailer: Apple Mail (2.3826.500.181.1.5) References: <72301E26-9618-462D-9C2F-F7CAC48517CA.ref@yahoo.com> X-Rspamd-Queue-Id: 4ZrlJF0ySbz3n6V X-Spamd-Bar: / X-Spamd-Result: default: False [-0.50 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; NEURAL_HAM_LONG(-0.99)[-0.990]; NEURAL_SPAM_SHORT(0.99)[0.985]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MLMMJ_DEST(0.00)[FreeBSD-pkg@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.146:from]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from] I know that pkg 2.1.2 and 2.1.0 and the use of pkg add are not yet whatever the long term intent is. The questions here are really about the long term intent. But, in formulating the question, I'll use information about existing pouriere(-devel) "bulk -c -a" behavior (or similarly large overall builds) when based on use of pkg 2.1.0 or later and the pkg add behavior for the build-depends, lib-depends, and run-depends stages of a package's build. Overall question: In the end, will any part of the process for "poudriere bulk -c -a" be worse than linear-time in the number of local packages? Why I ask . . . As stands, with pkg 2.1.0 or later involved, building the last 1/3rd of the packages appears to take longer than building the first 2/3rds of the packages. This illustrates the time taken for a package with dependencies is worse than linear in the number of local packages already built at the time. beefy18 (main-amd64-default, so: latest) pkg 2.1.0: 36494 35892 162 95 345 0 done: Tue, 15 Apr 2025 01:22:32 GMT 292:07:47 (That is over 3 times longer than when pkg 2.0.6 was in use. A debug kernel being involved seems to contribute notably to the scale factor. But it is a notable factor for non-debug kernels as well.) In progress with pkg 2.1.2 in use, 2 example points: 36512 24889 86 114 339 11084 parallel_build: Tue, 29 Apr 2025 01:22:26 = GMT 155:58:19 36512 22222 79 ??? ??? 13758 parallel_build: Tue, 29 Apr 2025 01:22:26 = GMT 134:30:55 For an example of 2.1.0 vs. 2.1.2 times for 134amd64-default (so, the non-debug kernel instead), pkg 2.1.2 then pkg 2.1.0 for beefy16: 36505 35999 80 104 322 0 done: Thu, 24 Apr 2025 07:24:46 GMT 159:17:19 36486 35978 72 110 326 0 done: Sat, 12 Apr 2025 01:05:21 GMT 167:34:49 So: similar time times frames. I expect similar to 292:07:47 for main-amd64-default with pkg 2.1.2 in use as well. But it takes a long time to have the specific evidence. =3D=3D=3D Mark Millard marklmi at yahoo.com