From nobody Tue Nov 30 03:04:41 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7928618AD514 for ; Tue, 30 Nov 2021 03:04:56 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-8.consmr.mail.gq1.yahoo.com (sonic307-8.consmr.mail.gq1.yahoo.com [98.137.64.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4J36X726n2z4t4P for ; Tue, 30 Nov 2021 03:04:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1638241495; bh=kDJNX2YdAYDXBPKcykuROA8eR97MU9d/Kik37+ksowk=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=HTPjL6+tdwMyvYc7WiakK5nywxOIg2a8h78fuc6O7hYZWFn0bYf53qjkOUw2TvcK+tAu1FGqARgypBV2Pu6Wznt22G+dAkQDJNmrfiSztZEOqxUTQhC9UpuujEwUCNzBTlKUvhUfrpOrvJkSRx4A2I7UCT1LkprOEfnR3rqH8xbf3yeq8pzcK4jYFOCflMu17Fc473iWqbWJkwM4m04Malz77TdzrjxHgPTXfr5ErFKfAfDHIpG3uAxTHXCDuPIbcFAf/hu1ZdpDQySTq17ibphmmyb7kZPL3qvpu8xEyVdUfB+og0r+a4x+mslHz91N86yEdXlKtGYosZrGPYNg0g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1638241495; bh=dixS/u2Z/bJkUhxRPr0+pU65MsdmgKJFro0ZuuRqV9n=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=N6ZOhv8ZjoezrssGGwGbgEHnYd6y6tvZW0YEt7DI51MsIv/fMXMQhKhDsuSTJAqu7bIrWxZARsYdan5tRFmfQQ4AMujFLRhE7PVmcRPnhjzIotwfIwHrT85NPg0B5K0G7iBWeqTJ+DQ7HvbaZKAqdLPaJKKM5BHNPvizUpD37vMonvBLrMJJuzZXO1OPscy1nw+VDx4twa0kgN81V8EKQzDcCI6k93aSyD+GYpn+6Nlem5ytM47MLd2SC2J6eQngyXv18IAJT7cRGEhChkAIOm0SbNjZ6218/0XIL4So6QMFApfVqi/C+NkS4NJGOHcfHD5jj4UE3L+6oEptF4FClA== X-YMail-OSG: Tj5WemIVM1nzKJ9TN6oFfSYfuEaxbfoU_Odphc6W8kI6wXuvrdYDXht1KJVEEpl WXRsZ4Nz6nqIAus_pmMr0apD8g6841ZPUJegPFPB_RK_b6MSH.mirYbTOR3kPNsMSxJatHn0sWZG fqghrYsFW2HH1wru0cv5hLzbEdNozbKxmAL3Tggg3ztBfWDpnK3r12uTwEsZdx6qDGliwTmCAxKj 2qEVllc7LnRm78h67TW8a8Kwb8.hFkL4WS_W6KznMb0BFPsFG0_VLkuV_WahqN0fq5h_3i6GtIRi wGpu2cqSg_NzgC1YzCCvahFQrYptfLC6FYNTjNpgsfFllooriGHGRpySXR3b2nxXvhMv_Qxrro0h CaN.BIn.yQBUycr5YPpPSXcfQSupmdxlOQ3qD5cdhjLyyE3HZFFvaebULOpU2RQ3ViTcXt7r9Ao0 PtCRfU0kyk.mVAWYPX870LZYdTc8yoKAi42_6xSU5b8kxosTcZTVWiMU17FN9m2JWZ7RrUoiXoOn XDEVrB5PnWLZ8_.F5J_pk7V6XOcy5snPHGlsCFzVIGg4MSDZPU03OwUnTXcgd2yOPaiqNLsNd2iU .N.WBQaIn37AA0oZtt8YHvVKiArMyhLIccyEpxdgRX71xqTdSPWrjeEnTv67dOcweewZebV9wkWd _LfcIzESGZwqVXxhUxppMLpKail0aBmSJoLiSksPy9V4GYiStqr_WjshC1hWOskEO827VY.l9qRC pmjuDyqfip1InDiydedBqys9MfDVDpKwSRs2WasAOKm98BTC9Rwael8wF013RexxO1uKe8graUVD YLMlrxQBzrtfxjSHuhcJmqdGcP0nokZIAFBRXTNtyZCJSBzBrwWYTyvpumuMfPCRvzyZ1vfJ5Ozf CM0P.YDCtwuiDp0wh72ZaRllRVhnzoOKrOP1oTpeqFkgx1lU2Wh_yq847KKFScakVq5HaHalSyeN nKNIX43fvF7g4m_BGqRjV2a5G8Rh.oQO7R4.qc.UFQvTZ4kCZZUtE1hwovxwFYNL.xzU.q8ZGH8l gt4tKZYqdLVunmYJfypk20DKyK9WFng0gDNQ5FHeAsu.YtbSPGmmTk_H_27E2nvEZEHqLBLR4wS_ hQyP4RozC9jk1RLJpfIEH2TKpUDRDp7FbWhNle9nj4lPeqm9NpS6cO31H7WvJ6zgiP88bZfbQK.T 7lj.H3GntnH9umopHUGGDgmUQkyttJxMauNCuLMecEJlMNsY6N3F4GTaDwqQHwHL8JtjDzfuq9ox yTKE.ffpN995jW0PffSgtJYRDmaWh345WJpC6NF.T1UVEndcAnFMeT0tSSyaWGtGrT.J2RaZeWv_ zG8QUXvJg9GCXvVfvjVLcB79GGG47oASWblUhjMDgBSM0WmYHeRlzLxKS2y6W2xiDRQtzaZozVPY dB0iwxJ1VZNr5Fvj6TQWqXP37Q3XEDX8Ss.bjZ7.33db3f2G4aMzeW5ObUGslF3cge1G6f21yirN WY.TnTXLNCAFfUuwvXDAE4ep6EcvmlZct4Nsqk2LOERV057pVuXTJN6zTZCgcn6ZGihdqpnCBwbw tAP7pcmqvh6JqBx75db7PGPrLVg2mJe0kXZoWXOnl.1V10yuz0BzKXdFIFrw1YJZdI954lLxrDzL c0hriR4OF3rtTLYpZXLGWold97WgHfrtu0d2GbJ5qj0iumByO5kU32..mQqHjfCnUPJTaHBP3BOF bB7biWVl6NtvBOL0CBM7T1wUA4H18.xg_KNLMu0CXF7WaKr0KRoOaIQF.D8el4qs5gvxyYAEszMC YEY9CngUOixfnN4MdyQRx1i8pivCzmoDI4l7wgqF6p50xYbKUCG8LVArvRcCDcB0SyzZaGn7YJuf N7M3etOBOpocia6T2EXxJv_lGPNO4YrxbkRWxA4CyfDLthL2sJrGQOBhCrILVciDTIgaslBWYd2p WsCSIHKZ5NjZ5Iy_25SAME3G..WD2YZ.fAICIJXW44ZLnm6nshV8Q0kfvfBBx6qZfpVoUVdt9t80 JwX4Pcz0A7aPbCUgbHN8i1BEEdR0_T7yBfaHME_0.BwNe9CNsTBDgip3ZttUSnXOm1s_8Lho4Nf3 PK72GJ6ew_pwuOFR8.5fhES33wfnbWTO77SjZlHLvjHOkfmm94IFE.gSbwW.S4DF8wbsxiOnB2xj Um03iriMDJLzW6vfMvMpPcVVbkWK10nqHFfXthKTnEeFeu5EXZEsH1YaJAStbtblGQMZz8T9iPvw U6r8oVOYhSkeWeaI4rMA4HcLP7wGSNoQfPonPXLE- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Tue, 30 Nov 2021 03:04:55 +0000 Received: by kubenode519.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 8b3b2304fe27918fbb360284e2fd11fe; Tue, 30 Nov 2021 03:04:43 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Call for Foundation-supported Project Ideas (buildworld buildkernel time issue) Message-Id: <896380E9-C3BB-4677-8CA8-CCCB3C5C94D8@yahoo.com> Date: Mon, 29 Nov 2021 19:04:41 -0800 To: Steve Kargl , FreeBSD Hackers X-Mailer: Apple Mail (2.3654.120.0.1.13) References: <896380E9-C3BB-4677-8CA8-CCCB3C5C94D8.ref@yahoo.com> X-Rspamd-Queue-Id: 4J36X726n2z4t4P X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=HTPjL6+t; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.17 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.32:from]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.67)[-0.670]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.32:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-hackers X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N From: Steve Kargl wrote on Date: Sun, 28 Nov 2021 14:07:32 -0800 > 1.) Replace clang with something/anything that is more performant. > Going on day 3 of "make buildworld". Still in the lib/clang/libclang > directory. Just an FYI for comparison: An appropriately configured 8GiByte RPi4B builds such in much less time than that: under 10 hours. Building the system llvm materials is included in the measured example below, but not a bootstrap compiler or linker. (This is the type of build example I give below because it was also handy for something I want to do.) I'd not call a well-configured 8 GiByte RPi4B high-end these days. But, nor is it low end as far as small board computers go. (Hardware like the MACCHIATObin Double Shot [4 Cortext-A72 cores, 16 GiBytes of RAM installed] and the old OverDrive 1000 [4 Cortext-A57 cores, 8 GiBytes of RAM installed]=20 are/were not SBCs and take/took noticeably less time based mostly on a more performant RAM + RAM-caching implementation from what I've seen. The slower clock rate and older Cortex variant in the OverDrive 1000 historicially took the least time of the 3, again mostly for RAM + RAM-caching tied performance reasons from what I saw.) The following is for a from-scratch debug build of main [so: 14] being built by a non-debug system that was built from the same source. Thus the WITH_META_MODE=3D that is in use adds some overhead to the specific build. It is an example where the system compiler and linker are built only once: bootstrapping copies are not built. That would add some time but is not needed often. (I've no clue if your 2+ day build built a bootstrap compiler and/or linker or not.) --- buildworld --- make[1]: "/usr/main-src/Makefile.inc1" line 340: SYSTEM_COMPILER: = Determined that CC=3Dcc matches the source tree. Not bootstrapping a = cross-compiler. make[1]: "/usr/main-src/Makefile.inc1" line 345: SYSTEM_LINKER: = Determined that LD=3Dld matches the source tree. Not bootstrapping a = cross-linker. It is a -j4 build (there are 4 cores in the RPi4B). buildworld time: World build completed on Mon Nov 29 18:12:55 PST 2021 World built in 23919 seconds, ncpu: 4, make -j4 So: somewhat under 6.7 hours. buildkernel time: Kernel build for GENERIC-DBG-CA72 completed on Mon Nov 29 18:40:44 PST = 2021 Kernel(s) GENERIC-DBG-CA72 built in 1669 seconds, ncpu: 4, make -j4 So: somewhat under 0.5 hours. Total time: 23919 sec + 1669 sec =3D=3D 25588 sec So: somewhat under 7.2 hours, but say under 10 hours to allow for some variation in what might be built and the like. For reference for the building environment: # uname -apKU FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #22 = main-n250972-319e9fc642a1-dirty: Tue Nov 23 12:25:36 PST 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400042 1400042 Even my "nodbg" builds include debug information, despite being optimized. It is the kernel's debug features which have been disabled. I give the src.config configuration later. The various WITHOUT_LLVM_TARGET_*'s do save some time but not huge amounts of it relative to the times reported here --but I also do WITH_CLANG_EXTRAS=3D which adds some time. I buildworld and buildkernel with -mcpu=3Dcortex-a72 involved, a type of thing I only do for lower end systems, not for something like a ThreadRipper 1950X. The build never used the swap space. My patched top (that tracks and reports various maximum-observed figures) reported: . . . Mem: . . . 2380Mi MaxObsActive, 3866Mi MaxObsWired, 4941Mi = MaxObs(Act+Wir+Lndry) . . . Swap: 14336Mi Total, 14336Mi Free, 2380Mi MaxObs(Act+Lndry+SwapUsed), = 4941Mi MaxObs(Act+Wir+Lndry+SwapUsed) (UFS tends to get very different Wired figures, and, so, also difference for various other figures.) The 8 GiByte RPi4B is using USB3 portable SSD media (a: T7 Touch). The media that I used is set up with root-on-ZFS (no UFS use) but historically root-on-UFS (no ZFS use) has not been a large variation. I could time via the UFS-based media if it is of interest (also T7 Touch media). The RPi4B has heat sinks and case with a fan. I use a CanaKit 5.1V 3.5A power supply. I have: over_voltage=3D6=20 arm_freq=3D2000=20 sdram_freq_min=3D3200=20 force_turbo=3D1=20 in the RPi4B's config.txt . These settings are ones that were set to work well with every RPi4B that I've used, with some margin. (All have heat sinks, a case with fan, and a 5.1V 3.5A power supply, so I've not tested other contexts.) The src.conf sort of material looks like: # more ~/src.configs/src.conf.CA72-dbg-clang.aarch64-host TO_TYPE=3Daarch64 TOOLS_TO_TYPE=3D${TO_TYPE} # KERNCONF=3DGENERIC-DBG-CA72 TARGET=3Darm64 .if ${.MAKE.LEVEL} =3D=3D 0 TARGET_ARCH=3D${TO_TYPE} .export TARGET_ARCH .endif # #WITH_CROSS_COMPILER=3D WITH_SYSTEM_COMPILER=3D WITH_SYSTEM_LINKER=3D # #WITH_LLD_BOOTSTRAP=3D WITH_ELFTOOLCHAIN_BOOTSTRAP=3D #Disables avoiding bootstrap: WITHOUT_LLVM_TARGET_ALL=3D WITH_LLVM_TARGET_AARCH64=3D WITH_LLVM_TARGET_ARM=3D WITHOUT_LLVM_TARGET_MIPS=3D WITHOUT_LLVM_TARGET_POWERPC=3D WITHOUT_LLVM_TARGET_RISCV=3D WITHOUT_LLVM_TARGET_X86=3D #WITH_CLANG_BOOTSTRAP=3D WITH_CLANG=3D WITH_CLANG_IS_CC=3D WITH_CLANG_FULL=3D WITH_CLANG_EXTRAS=3D WITH_LLD=3D WITH_LLD_IS_LD=3D WITH_LLDB=3D # WITH_BOOT=3D # # WITHOUT_WERROR=3D #WERROR=3D #MALLOC_PRODUCTION=3D WITHOUT_MALLOC_PRODUCTION=3D WITH_ASSERT_DEBUG=3D WITH_LLVM_ASSERTIONS=3D # # Avoid stripping but do not control host -g status as well: DEBUG_FLAGS+=3D # WITH_REPRODUCIBLE_BUILD=3D WITH_DEBUG_FILES=3D # XCFLAGS+=3D -mcpu=3Dcortex-a72 XCXXFLAGS+=3D -mcpu=3Dcortex-a72 # There is no XCPPFLAGS but XCPP gets XCFLAGS content. ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a72+crypto ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto (Comments about why specific options were not used for reasons of some odd consequence once observed may not have been checked in some time. Options commented out without such notes are just a simple choices, not driven by such oddities.) One thing that can slow down builds if there is rapid build output at times: serial console handling of that output. (Very noticeable for installworld and installkernel to a directory.) I used an ssh session to avoid the potential contribution to the time. The OverDrive 1000 died some time ago but I still have access to the MACHHIATObin Double Shot and I could run a timing test on it for building the same sources that same way. (Same Cortex-A72 clock rate in use as used in the RPi4B test: 2.0 GHz.) Hmm. The buildkernel got a bunch of: ERROR: ctfconvert: failed to get mapping for tid ????? notices. I do not expect the issue changed the time much but note them in case. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)