From nobody Fri Jun 10 03:34:47 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5C2EB83C0E4; Fri, 10 Jun 2022 03:34:51 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LK66252tpz3s7g; Fri, 10 Jun 2022 03:34:50 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mail-wm1-x335.google.com with SMTP id z17so8450377wmi.1; Thu, 09 Jun 2022 20:34:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:date:to:subject:message-id:mime-version; bh=maDiKSMR6Ns4OlvXx+ONmTL/nkRWb1DzTq+VV6jPmIM=; b=ieTqFmXF7hiZVJ/NycIdvLetvfXfsc52XaTdqer1EAYMw1S6e2w5x1CW5GwYOXEgdO Xs3+aKYzmwlAy2WNl80kstBzT62hkpWDb+ep5yVrFc1/ZlFw31VLAEBpsZktNWaD/3nr AK9cwI5w5dUaeOmHSU+4/PaIvmwzQa1FiAKWIyEDBJkYuL8Qn2rX1vW+91NxMVNUJrpU dq30oIuLO2EDzS1qEF5bVa1LS2w+RISh1w4WyafQHrurF1x36b7PA2+sDT7AjRwZh5DL /z2KMb8NQ3tVkqkMF/q3F4NSZWp7GUVFsjU+mTFSmoyJxYCrS4hsWRnPHxQSTYtvP4WI kQoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:date:to:subject:message-id:mime-version; bh=maDiKSMR6Ns4OlvXx+ONmTL/nkRWb1DzTq+VV6jPmIM=; b=8CO4rFhnWGTuoNpUbqJR8dfzoa/uUdRHducsfT3w0GqIsx7uMjtHj94vpRlcdtyTYU w9GerCFIb3+2Nb6DW6B7PpSxVLG1ziiskdwU3FWq3+qduLCubnnGH9WGZOhEPfBlS6BB 6pvQnhEBR/jjI0rSSKNU+8Fw4NglUUMT9BjafyuhBwPeGuwj8j4VC9KNsuRSi4QXjKXh zT12BDAzf4qx1QDjAQ49lZQGM8ZqHMpOx/uDmePXjfNJ/DFigwzzMmk/vJ488ZKG1qHr M+m72SEu1QQEZI6/vtoDc3t9qvgnJf1E9Fu3KDIkN/pofH2d9dyZCqXbK6Hav0wnmkM7 qtjw== X-Gm-Message-State: AOAM5324N0kRAt/WQS1r7WaX1fc/1ke8CScZT760aKjsfuvZGSqSaj8s /TRzvfxASDj66ypgOCYRp7IFxFiuPA4= X-Google-Smtp-Source: ABdhPJyU5q0D6+6n8vTO7/HrQyloRGB0H0ZMGMNsH1cW/TOrR1gUFkFhZs4a3Dqr9WqWwQcS3HGDJA== X-Received: by 2002:a05:600c:3d18:b0:39c:474c:eb with SMTP id bh24-20020a05600c3d1800b0039c474c00ebmr6368824wmb.87.1654832089637; Thu, 09 Jun 2022 20:34:49 -0700 (PDT) Received: from rimwks.local ([2001:470:1f15:3d8:9554:7133:29d0:f593]) by smtp.gmail.com with ESMTPSA id v187-20020a1cacc4000000b0039749256d74sm1317452wme.2.2022.06.09.20.34.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jun 2022 20:34:49 -0700 (PDT) From: Rozhuk Ivan X-Google-Original-From: Rozhuk Ivan Date: Fri, 10 Jun 2022 06:34:47 +0300 To: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org, Rozhuk Ivan Subject: ccache during system build Message-ID: <20220610063447.10ff8004@rimwks.local> X-Mailer: Claws Mail 3.19.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.1) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/EaWb/ZuBLx5HcyfQ9sZFz/p" X-Rspamd-Queue-Id: 4LK66252tpz3s7g X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=ieTqFmXF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rozhukim@gmail.com designates 2a00:1450:4864:20::335 as permitted sender) smtp.mailfrom=rozhukim@gmail.com X-Spamd-Result: default: False [-2.29 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; HAS_ATTACHMENT(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.987]; FREEMAIL_TO(0.00)[freebsd.org,gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~,3:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; MIME_UNKNOWN(0.10)[application/x-shellscript]; RCPT_COUNT_THREE(0.00)[3]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_BAD_ATTACHMENT(1.60)[sh]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::335:from]; MLMMJ_DEST(0.00)[freebsd-current,freebsd-hackers]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --MP_/EaWb/ZuBLx5HcyfQ9sZFz/p Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi! Just few tips. 1. bsd.compiler.mk set COMPILERCHECK env var: ============================================================================ # Handle bootstrapped compiler changes properly by hashing their content # rather than checking mtime. For external compilers it should be safe # to use the more optimal mtime check. # XXX: CCACHE_COMPILERCHECK= string: .if ${CC:N${CCACHE_BIN}:[1]:M/*} == "" CCACHE_COMPILERCHECK?= content .else CCACHE_COMPILERCHECK?= mtime .endif ============================================================================ this overrides ccache.conf setting and increase build time. CCACHE_COMPILERCHECK='%compiler% -dumpversion' makes first build and second system rebuild faster. 2. To allow ccache hit in case with different src and obj location a) obj must be placed inside src b) OBJTOP env var must be set to new obj location without this obj path will contain src path and not hit to ccache. c) CCACHE_BASEDIR must be set to src d) CCACHE_NOHASHDIR=yes e) CCACHE_COMPILERCHECK='%compiler% -dumpversion' f) MAKEOBJDIRPREFIX='' g) -o nocache must be used with mount_nullfs to avoid error during install: exec(btxld) failed (No such file or directory) and obj via nullfs As example, attached scripts build kernel and world using obj stored inside /tmp, after build files from temp obj moved to /usr/obj dir. Using simular technic for build FreeBSD inside other src and obj locations I got ccache hit and reduce build time. I got ccache miss only for kernel build in case different KERNCONF, because it is mapped to obj dir path. Probably overwriting OBJROOT will fix it. --MP_/EaWb/ZuBLx5HcyfQ9sZFz/p Content-Type: application/x-shellscript Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=fbsd_build_kernel.sh IyEvYmluL3NoCiMjIyBSb3podWsgSXZhbiAyMDE5LTIwMjIKIyMjIGZic2RfYnVpbGRfa2VybmVs CiMjIyAKCgojIEV4aXQgb24gZXJyb3IuCnNldCAtZQoKCiMgSW5pdCBjb25zdGFucy4KOiAke1NS Q19ESVI9L3Vzci9zcmN9CkFVVE9fVEhSRUFEUz1gZ2V0Y29uZiBOUFJPQ0VTU09SU19PTkxOIHwg dHIgLWNkICdbOnByaW50Ol0nYApNQUtFT0JKRElSX09SSUc9YG1ha2UgLUMgJHtTUkNfRElSfSAt ViBNQUtFT0JKRElSYApPQkpESVJfU1JDPSIke1NSQ19ESVJ9L29ial90bXAiCk9CSkRJUl9UTVA9 YG1rdGVtcCAtZCAtcSAvdG1wL29iai5YWFhYWFhYWGAKaWYgWyAkez99IC1uZSAwIF07IHRoZW4K CWVjaG8gIiR7MH06IENhbid0IGNyZWF0ZSB0ZW1wIGRpciwgZXhpdGluZy4uLiIKCWV4aXQgMQpm aQpleHBvcnQgQ0NBQ0hFX0JBU0VESVI9IiR7U1JDX0RJUn0iCmV4cG9ydCBDQ0FDSEVfTk9IQVNI RElSPSd5ZXMnCmV4cG9ydCBDQ0FDSEVfQ09NUElMRVJDSEVDSz0nJWNvbXBpbGVyJSAtZHVtcHZl cnNpb24nCmV4cG9ydCBNQUtFT0JKRElSUFJFRklYPScnCmV4cG9ydCBPQkpUT1A9IiR7T0JKRElS X1NSQ30iCmlmIFsgISAtZCAiJHtPQkpESVJfU1JDfSIgXTsgdGhlbgoJcm0gLXJmICIke09CSkRJ Ul9TUkN9IgoJbWtkaXIgLXAgIiR7T0JKRElSX1NSQ30iCmZpCgoKIyBDbGVhbiBhbGwgcHJldiBk YXRhLgpmb3IgX19NTlRfUE9JTlQgaW4gYG1vdW50IHwgZ3JlcCAiIG9uICR7T0JKRElSX1NSQ30i IHwgYXdrICd7cHJpbnQgJDE7fSdgIDsgZG8KCXJtIC1yZiAiJHtfX01OVF9QT0lOVH0iID4vZGV2 L251bGwgMj4mMQoJdW1vdW50IC1mICIke09CSkRJUl9TUkN9IiA+L2Rldi9udWxsIDI+JjEKZG9u ZQoKCiMgLW8gbm9jYWNoZSB0byBmaXggZXJyb3I6IGV4ZWMoYnR4bGQpIGZhaWxlZCAoTm8gc3Vj aCBmaWxlIG9yIGRpcmVjdG9yeSkKbW91bnRfbnVsbGZzIC1vIHJ3IC1vIG5vY2FjaGUgLW8gbm9h dGltZSAtbyBub2V4ZWMgLW8gbm9zdWlkICIke09CSkRJUl9UTVB9IiAiJHtPQkpESVJfU1JDfSIK CgojIEJ1aWxkLgovdXNyL2Jpbi9uaWNlIC1uIDE1IFwKCS91c3IvYmluL3RpbWUgLWggXAoJbWFr ZQktaiIke0FVVE9fVEhSRUFEU30iIFwKCQktQyAiJHtTUkNfRElSfSIgXAoJCS1zIFwKCQktREFM V0FZU19DSEVDS19NQUtFIFwKCQktRE5PX0NMRUFOIFwKCQlidWlsZGtlcm5lbAojIEZvcmNlIHNl dCBmaWxlcyB0aW1lcy4KZmluZCAiJHtPQkpESVJfVE1QfSIgLXR5cGUgZiAtZXhlYyB0b3VjaCB7 fSArCgoKIyBDb3B5IGJhY2sgdG8gL3Vzci9vYmoKX19TUkNfRElSPSIke09CSkRJUl9UTVB9L3N5 cyIKX19EU1RfRElSPSIke01BS0VPQkpESVJfT1JJR30vc3lzIgplY2hvICJTYXZpbmc6ICR7X19T UkNfRElSfSAtPiAke19fRFNUX0RJUn0iCnJtIC1yZiAiJHtfX0RTVF9ESVJ9Igpta2RpciAtcCAi JHtfX0RTVF9ESVJ9IgpjaG1vZCBhK3JYICIke19fRFNUX0RJUn0iCmNobW9kIGErclggIiR7TUFL RU9CSkRJUl9PUklHfSIKY2htb2QgLVIgYStyWCAiJHtfX1NSQ19ESVJ9IgpjcCAtYWYgIiR7X19T UkNfRElSfS8iICIke19fRFNUX0RJUn0iCgoKIyBDbGVhbnVwLgp1bW91bnQgIiR7T0JKRElSX1NS Q30iCnJtIC1yZiAiJHtPQkpESVJfVE1QfSIKc3luYwoKCmV4aXQgMAo= --MP_/EaWb/ZuBLx5HcyfQ9sZFz/p Content-Type: application/x-shellscript Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=fbsd_build_world.sh IyEvYmluL3NoCiMjIyBSb3podWsgSXZhbiAyMDE5LTIwMjIKIyMjIGZic2RfYnVpbGRfd29ybGQK IyMjIAoKCiMgRXhpdCBvbiBlcnJvci4Kc2V0IC1lCgoKIyBJbml0IGNvbnN0YW5zLgo6ICR7U1JD X0RJUj0vdXNyL3NyY30KQVVUT19USFJFQURTPWBnZXRjb25mIE5QUk9DRVNTT1JTX09OTE4gfCB0 ciAtY2QgJ1s6cHJpbnQ6XSdgCk1BS0VPQkpESVJfT1JJRz1gbWFrZSAtQyAke1NSQ19ESVJ9IC1W IE1BS0VPQkpESVJgCk9CSkRJUl9TUkM9IiR7U1JDX0RJUn0vb2JqX3RtcCIKT0JKRElSX1RNUD1g bWt0ZW1wIC1kIC1xIC90bXAvb2JqLlhYWFhYWFhYYAppZiBbICR7P30gLW5lIDAgXTsgdGhlbgoJ ZWNobyAiJHswfTogQ2FuJ3QgY3JlYXRlIHRlbXAgZGlyLCBleGl0aW5nLi4uIgoJZXhpdCAxCmZp CmV4cG9ydCBDQ0FDSEVfQkFTRURJUj0iJHtTUkNfRElSfSIKZXhwb3J0IENDQUNIRV9OT0hBU0hE SVI9J3llcycKZXhwb3J0IENDQUNIRV9DT01QSUxFUkNIRUNLPSclY29tcGlsZXIlIC1kdW1wdmVy c2lvbicKZXhwb3J0IE1BS0VPQkpESVJQUkVGSVg9JycKZXhwb3J0IE9CSlRPUD0iJHtPQkpESVJf U1JDfSIKaWYgWyAhIC1kICIke09CSkRJUl9TUkN9IiBdOyB0aGVuCglybSAtcmYgIiR7T0JKRElS X1NSQ30iCglta2RpciAtcCAiJHtPQkpESVJfU1JDfSIKZmkKCgojIENsZWFuIGFsbCBwcmV2IGRh dGEuCmZvciBfX01OVF9QT0lOVCBpbiBgbW91bnQgfCBncmVwICIgb24gJHtPQkpESVJfU1JDfSIg fCBhd2sgJ3twcmludCAkMTt9J2AgOyBkbwoJcm0gLXJmICIke19fTU5UX1BPSU5UfSIgPi9kZXYv bnVsbCAyPiYxCgl1bW91bnQgLWYgIiR7T0JKRElSX1NSQ30iID4vZGV2L251bGwgMj4mMQpkb25l CgoKIyAtbyBub2NhY2hlIHRvIGZpeCBlcnJvcjogZXhlYyhidHhsZCkgZmFpbGVkIChObyBzdWNo IGZpbGUgb3IgZGlyZWN0b3J5KQptb3VudF9udWxsZnMgLW8gcncgLW8gbm9jYWNoZSAtbyBub2F0 aW1lICIke09CSkRJUl9UTVB9IiAiJHtPQkpESVJfU1JDfSIKCgojIEJ1aWxkLgovdXNyL2Jpbi9u aWNlIC1uIDE1IFwKCS91c3IvYmluL3RpbWUgLWggXAoJbWFrZQktaiIke0FVVE9fVEhSRUFEU30i IFwKCQktQyAiJHtTUkNfRElSfSIgXAoJCS1zIFwKCQktREFMV0FZU19DSEVDS19NQUtFIFwKCQkt RE5PX0NMRUFOIFwKCQlidWlsZHdvcmxkCiMgRm9yY2Ugc2V0IGZpbGVzIHRpbWVzLgpmaW5kICIk e09CSkRJUl9UTVB9IiAtdHlwZSBmIC1leGVjIHRvdWNoIHt9ICsKCgojIENvcHkgYmFjayB0byAv dXNyL29iagpfX1NSQ19ESVI9IiR7T0JKRElSX1RNUH0iCl9fRFNUX0RJUj0iJHtNQUtFT0JKRElS X09SSUd9IgplY2hvICJTYXZpbmc6ICR7X19TUkNfRElSfSAtPiAke19fRFNUX0RJUn0iCm1rZGly IC1wICIke19fRFNUX0RJUn0iCmNobW9kIGErclggIiR7X19EU1RfRElSfSIKY2htb2QgLVIgYSty WCAiJHtfX1NSQ19ESVJ9IgpybSAtcmYgIiR7X19TUkNfRElSfS9zeXMiCmZvciBfX0RTVF9TVUJf RElSIGluIGBmaW5kICIke19fRFNUX0RJUn0vIiAtbWF4ZGVwdGggMSAtbm90IC1uYW1lICcnYDsg ZG8KCVsgIiR7X19EU1RfU1VCX0RJUn0iID09ICIke19fRFNUX0RJUn0vc3lzIiBdICYmIGNvbnRp bnVlCglybSAtcmYgIiR7X19EU1RfU1VCX0RJUn0iCmRvbmUKY3AgLWFmICIke19fU1JDX0RJUn0v IiAiJHtfX0RTVF9ESVJ9IgoKCiMgQ2xlYW51cC4KdW1vdW50ICIke09CSkRJUl9TUkN9IgpybSAt cmYgIiR7T0JKRElSX1RNUH0iCnN5bmMKCgpleGl0IDAK --MP_/EaWb/ZuBLx5HcyfQ9sZFz/p--