From nobody Mon Sep 06 10:14:20 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4D88717AF200 for ; Mon, 6 Sep 2021 10:14:33 +0000 (UTC) (envelope-from wschnr@googlemail.com) Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H34551kBHz4tNP; Mon, 6 Sep 2021 10:14:33 +0000 (UTC) (envelope-from wschnr@googlemail.com) Received: by mail-lf1-f43.google.com with SMTP id s10so12518866lfr.11; Mon, 06 Sep 2021 03:14:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qq+BqNvYqWxJD9pY9HdLm0AIb+hMCFxPdB80G72FUzQ=; b=t1uI0N8jvQhV5ST0B3HoWmd4REgz01X4AC+WYc4+7xullbthoDp88qOAl/GSN0LgmT PjHg+ixGOEe7rHjui3xPZbD6N8YTjxtr35fvRoWWLJ4Hv2r0DN3m42kTY/6GiYxVuGKc s3+3omoJLtrOgkdPh9DuQwk2pl7MC/NH6Iu+GhyeW5kiQESexuui9lIc2XaSL//FuBLe dzt+Zg1N80WyjvpmRtQtQxO7HXwgsUBsdKbryKLeIIM6IQkkM9Y0aS8Qaeebvb+pS4ES nSZVZrpLuSYv027CuZDQa08hR9xd5XAEXstKSYQ7GEbMWGJMl+uJ9A4TEvDRqlyr669C J2Sg== X-Gm-Message-State: AOAM531sbAgidf1iJgPtndLt5TBONVstCFdUAv2uJOH8pJqug5tdgk7I VRD7I+3qHBSrfAf8exXUYaGFNxuHy+cWH1KV+i5k/U9utOo= X-Google-Smtp-Source: ABdhPJxNXXLzP2qk/f0hhAX+kG952AgnxqR+v0PYQKYUggpIZAqMr64QCeWqoawJ7XQ7AfbIcbAdHtseH2ajMW0oC5U= X-Received: by 2002:ac2:4e98:: with SMTP id o24mr8933852lfr.295.1630923271361; Mon, 06 Sep 2021 03:14:31 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Wolfram Schneider Date: Mon, 6 Sep 2021 12:14:20 +0200 Message-ID: Subject: Re: -CURRENT compilation time To: Jeremie Le Hen Cc: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4H34551kBHz4tNP X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, 6 Sept 2021 at 10:10, Jeremie Le Hen wrote: > > Hey, > > I want to build -CURRENT again from sources. It's been a long time > since I hadn't done that. I'm shocked by the compilation time. > > I started the whole thing on Friday night and Monday morning it's > still in stage 4.2 (building libraries). Through occasional glancing > at the screen over the weekend, it seems obvious to me that the > compilation time is utterly dominated by LLVM. Compiling C++ seems > extremely CPU heavy and this is made worse by the fact LLVM is built > twice (once for build/cross tools, once for the actual world). > > So OK, my CPU is not the most powerful out there but it's still decent [1]. > > So I have a couple of questions coming to my mind: > 1. Is there any optimization I could benefit from? (I'm sure there's a > knob to use the existing compiler instead of building a > cross-compiler.) > 2. More generally, isn't this compilation time not considered as a > problem for developers? This seems to terribly slow down the iteration > time for people working on the build system. I wouldn't be surprised > if this drove people away from working on/improving that area. > > [1] https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i5-6260U+%40+1.80GHz&id=2671 Hi, I ran buildworld yesterday on a 3 CPU VM (AMD EPYC 2.4GHz) and the runtime was 2h. We spent most of the time in "stage 4.2: building libraries", in my case 62% of the CPU time and 75% of the real time. I guess the build time on your laptop should be around 6 hours if everything is ok. time make -j $(sysctl -n hw.ncpu) buildworld > log.buildworld 2>&1 tail -n 5 log.buildworld -------------------------------------------------------------- >>> World build completed on Sat Sep 4 20:58:00 UTC 2021 >>> World built in 7235 seconds, ncpu: 3, make -j3 -------------------------------------------------------------- 7235.61 real 20527.30 user 915.88 sys egrep '>>> stage| real ' log.buildworld >>> stage 1.1: legacy release compatibility shims 0.28 real 0.18 user 0.10 sys >>> stage 1.2: bootstrap tools 165.99 real 472.58 user 11.56 sys >>> stage 2.1: cleaning up the object tree 21.47 real 36.96 user 14.14 sys 15.87 real 29.14 user 11.87 sys >>> stage 2.3: build tools 2.42 real 3.79 user 0.62 sys >>> stage 3: cross tools 9.92 real 18.49 user 1.75 sys >>> stage 3.1: recording build metadata 0.07 real 0.01 user 0.06 sys >>> stage 4.1: building includes 16.62 real 36.46 user 9.48 sys >>> stage 4.2: building libraries 5440.89 real 15724.60 user 482.58 sys >>> stage 4.3: building lib32 shim libraries 615.91 real 1654.77 user 164.58 sys >>> stage 4.4: building everything 937.23 real 2540.06 user 205.47 sys -Wolfram -- Wolfram Schneider https://wolfram.schneider.org