From nobody Tue Jun 27 09:09:54 2023 X-Original-To: freebsd-pkg@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QqzSN5LBMz4kPT9 for ; Tue, 27 Jun 2023 09:09:56 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from mail.rlwinm.de (mail.rlwinm.de [138.201.35.217]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QqzSN0Qrwz41kT for ; Tue, 27 Jun 2023 09:09:56 +0000 (UTC) (envelope-from crest@rlwinm.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of crest@rlwinm.de designates 138.201.35.217 as permitted sender) smtp.mailfrom=crest@rlwinm.de; dmarc=none Received: from [IPV6:2001:9e8:944:5600:5484:2f3f:3b02:a1e1] (unknown [IPv6:2001:9e8:944:5600:5484:2f3f:3b02:a1e1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail.rlwinm.de (Postfix) with ESMTPSA id DC77323D47 for ; Tue, 27 Jun 2023 09:09:54 +0000 (UTC) Message-ID: <74d8a1b7-45b7-e61a-4b82-fe7ebaf0aa02@rlwinm.de> Date: Tue, 27 Jun 2023 11:09:54 +0200 List-Id: Binary package management and package tools discussion List-Archive: https://lists.freebsd.org/archives/freebsd-pkg List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-pkg@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: Suggestion for speeding up pkg Content-Language: en-US To: freebsd-pkg@freebsd.org References: From: Jan Bramkamp In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.97 / 15.00]; NEURAL_HAM_SHORT(-0.92)[-0.917]; NEURAL_HAM_LONG(-0.89)[-0.894]; NEURAL_HAM_MEDIUM(-0.86)[-0.861]; R_SPF_ALLOW(-0.20)[+mx:c]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-pkg@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:24940, ipnet:138.201.0.0/16, country:DE]; DMARC_NA(0.00)[rlwinm.de]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pkg@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4QqzSN0Qrwz41kT X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N On 08.06.23 03:50, Patrick McMunn wrote: > I'm just a regular user of FreeBSD, but I'd like to make a suggestion. > When installing a large set of packages, such as "pkg install kde5", > it can take quite a while to download all that data. Why does pkg > download ALL the packages and only then begin to extract and install > them? Would it not be a more efficient use of time to calculate > dependencies first, then begin downloading the packages in the order > in which they are required, and then begin extracting and installing > them as they are downloaded instead of waiting for the download to > complete before beginning installation? > > I understand that part of the reason is verification and such, but the > default could be extracting and installing to a temporary directory > then merging to the live system once everything is downloaded and > extracted. Make the behavior configurable by the user. Set sane > defaults. Check for sufficient disk space immediately after dependency > calculation to make sure space is available for the downloads, the > extracted contents, and any other necessities. This will make for a > more pleasant user experience. You just dismissed multiple new failure cases that could leave users stuck with a partly upgraded system. In my opinion fetching the latest version of all packages is the right thing to do, but the public package mirrors are painfully slow for some users. How are you fetching your packages? What transfer speeds are did you encounter? How long did fetching all dependencies take?