From nobody Wed Feb 15 19:50:59 2023 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PH7x24ZBkz3rGK8 for ; Wed, 15 Feb 2023 19:51:02 +0000 (UTC) (envelope-from SRS0=oYLu=6L=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PH7x16Zncz4M7T for ; Wed, 15 Feb 2023 19:51:01 +0000 (UTC) (envelope-from SRS0=oYLu=6L=quip.cz=000.fbsd@elsa.codelab.cz) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of "SRS0=oYLu=6L=quip.cz=000.fbsd@elsa.codelab.cz" has no SPF policy when checking 94.124.105.4) smtp.mailfrom="SRS0=oYLu=6L=quip.cz=000.fbsd@elsa.codelab.cz"; dmarc=none Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 8413FD7892 for ; Wed, 15 Feb 2023 20:51:00 +0100 (CET) Received: from [192.168.145.50] (ip-89-177-27-225.bb.vodafone.cz [89.177.27.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id BDB6FD7891 for ; Wed, 15 Feb 2023 20:50:59 +0100 (CET) Message-ID: <97cefec2-d97a-4151-c113-6046e9bb5ea0@quip.cz> Date: Wed, 15 Feb 2023 20:50:59 +0100 List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 To: FreeBSD Mailing List Content-Language: cs-Cestina, en-US From: Miroslav Lachman <000.fbsd@quip.cz> Subject: How poudriere's PACKAGE_FETCH_WHITELIST should work? Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-0.80 / 15.00]; AUTH_NA(1.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=oYLu=6L=quip.cz=000.fbsd@elsa.codelab.cz]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-ports@freebsd.org]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; R_SPF_NA(0.00)[no SPF record]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; R_DKIM_NA(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DMARC_NA(0.00)[quip.cz]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ports@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=oYLu=6L=quip.cz=000.fbsd@elsa.codelab.cz] X-Rspamd-Queue-Id: 4PH7x16Zncz4M7T X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N Poudriere and fetching build dependecies - I would like to know how it is supposed to work? I use poudriere a bunch of years, with ports overlays etc. As I need to rebuild packages for our machines many times a month I am tired of building huge and slow packages like llvm, gcc, rust (as they often eat all memory+swap then build is killed by OOM). I tied to setup PACKAGE_FETCH with the two following variables in poudriere.conf PACKAGE_FETCH_BRANCH=quarterly PACKAGE_FETCH_WHITELIST="gcc* rust llvm* gcc90 gcc10 gcc11 gcc12 gcc13 gcc14 llvm10 llvm11 llvm12 llvm13 llvm14 llvm15 lua54" When I started usual "poudriere bulk" only gcc12 were fetched, llvm10 and rust built from sources: [00:01:02] Package fetch: Will fetch 1 packages from remote or local pkg cache The following packages will be fetched: New packages to be FETCHED: gcc12: 12.2.0_5 (81 MiB: 100.00% of the 81 MiB to download) Number of packages to be fetched: 1 I thought maybe I have some different options selected for llvm and rust than the default on official FreeBSD packages, I double checked, removed stored options and started another poudriere bulk with different package set (llvm10 and rust will be needed for the set). This time the rust package was downloaded, but llvm10 built from source again: [00:00:22] Package fetch: Will fetch 1 packages from remote or local pkg cache The following packages will be fetched: New packages to be FETCHED: rust: 1.66.0 (112 MiB: 100.00% of the 112 MiB to download) Number of packages to be fetched: 1 The process will require 112 MiB more space. 112 MiB to be downloaded. [xxxxx] Fetching rust-1.66.0.pkg: 100% 112 MiB 39.2MB/s 00:03 But the mystery is that "poudriere bulk" failed on building rust even if it should be used from fetched package: [04:11:44] Failed ports: lang/rust:build [04:11:44] Skipped ports: devel/cargo-c graphics/libimagequant graphics/py-pillow@py37 I checked the logs and the rust build process was killed by OOM killer. (but why poudriere was building it if it was already fetched?) I started poudriere bulk again, rust was fetch again a this time it was really used to build other packages, no rebuild of rust from sources needed. I am really confused why Poudriere fetches only 1 package at a time if it should fetch gcc12, rust and llvm10? Why Poudriere tried to build rust if it fetches it as pkg? And why it does not fetch llvm10 even if it is available and we do not have options stored for devel_llvm10? Kind regards Miroslav Lachman