From nobody Wed Sep 08 12:47:11 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 69BB117B3A60 for ; Wed, 8 Sep 2021 12:47:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-vs1-xe2a.google.com (mail-vs1-xe2a.google.com [IPv6:2607:f8b0:4864:20::e2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H4MNd2M1rz3jyT for ; Wed, 8 Sep 2021 12:47:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-vs1-xe2a.google.com with SMTP id u1so1881120vsq.10 for ; Wed, 08 Sep 2021 05:47:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=kKpOSByUqJ1dvhI5Hrmq0LSlIc/yGFemZYP9Ei9g5Tc=; b=iwTdfBp4chSEJ2JAdr2hUBt2SuH4QoTEkujNKoqClrtq0KUVHrKb2tr8AKXO4+owrt rCMDLKZB48ziwQl6Z4o5Kiwl2YsuecFEFe0JFQvedhnrn5TBP8W4laUpwSBEL3cvYZj/ 6Uzrr1PJGVMjLqfF6DOAFmk69EMtbiwtS3KAcIdtdGLiLTUEjzRYN7sMbuxGj51f/5L1 f2uC0NLPGHU3l89amvEWdHNSvI//y/fT0iPCJKn1jMSEzPs1eFL2NLCWpXETEEebiWhS dm2wFKV6o08wdLfhvhubPC0TKaKwWVJZakreCcp+95mqGxKDZg1kOoHbHF40YHOU0uUk ND7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=kKpOSByUqJ1dvhI5Hrmq0LSlIc/yGFemZYP9Ei9g5Tc=; b=YkQYmwVFL9YRBQNkXM7dSV3qXRgYVXT6Gj05SR6ZtRN31RLgs/hg+bkEso2y+PmNwB 32lzhJ8LlRKfriNPV+vEWbXLM1XB2WG5jO2MiG9+bXoYat3N717g9gcV3Qh81tDfycrN x99sEby/liuaC4T2dOhPI+qwv8e1sId1GYBDbjfra5CHlo5p5h7kViMYVQmgyS3BZrHp nSBQoxPwTZGRsjGcg294SswKRWDhgVIXAfk7saopPLKTZUpbhonfdjpMYFPgZx6jT/w1 wXtOdeDBCdwQh4lInIvQzDiNd4RX/eoKqfG5E7wvoSEbPMOR6837KckL2LHgexrrlU8U ppUg== X-Gm-Message-State: AOAM530b/Lcg5yBAOJUAVonqvHppgqyK4ZQ6UbOKYBnJaeNhYDS/ZIq+ X84tBsLb1Yu4OeCgbxif3uTgOu9PrAB3S0mXOPS0+Iu+S+GCQ3j0 X-Google-Smtp-Source: ABdhPJzU+61K7azshQugufkYmEdwdkJ+ARaxen9oBIuj37jGjLEAC51CUnjzAiVj0aB+UzjNXnKIePhtsQ/zgOKXhI0= X-Received: by 2002:a05:6102:2f6:: with SMTP id j22mr1747720vsj.12.1631105242913; Wed, 08 Sep 2021 05:47:22 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <2cfb912a-618b-9f06-9cef-d2fe1d78fe97@FreeBSD.org> In-Reply-To: <2cfb912a-618b-9f06-9cef-d2fe1d78fe97@FreeBSD.org> From: Warner Losh Date: Wed, 8 Sep 2021 06:47:11 -0600 Message-ID: Subject: Re: -CURRENT compilation time To: David Chisnall Cc: Stefan Esser , FreeBSD Current Content-Type: multipart/alternative; boundary="00000000000098786c05cb7b4836" X-Rspamd-Queue-Id: 4H4MNd2M1rz3jyT X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --00000000000098786c05cb7b4836 Content-Type: text/plain; charset="UTF-8" On Wed, Sep 8, 2021 at 2:59 AM David Chisnall wrote: > On 07/09/2021 18:02, Stefan Esser wrote: > > Wouldn't this break META_MODE? > > I have never managed to get META_MODE to work but my understanding is > that META_MODE is addressing a problem that doesn't really exist in any > other build system that I've used: that dependencies are not properly > tracked. > It does track the dependencies. It uses filemon(4) to do this so that every dependency is tracked. This is better than what other build systems do because no dependencies are missed. > When I do a build of LLVM with the upstream build system with no > changes, it takes Ninja approximately a tenth of a second to stat all of > the relevant files and tell me that I have no work to do. META_MODE > apparently lets the FreeBSD build system extract these dependencies and > do something similar, but it's not enabled by default and it's difficult > to make work. > META_MODE does the same thing, and it's just as fast as ninja make. And it has the advantage that, unlike meson, it isn't rebuilding the makefiles all the time. And apart from loading filmon, it was quite easy to enable last time I was using it (though I did have trouble finding the right docs). > > I'd rather be able to continue building the world within a few minutes > > (generally much less than 10 minutes, as long as there is no major LLVM > > upgrade) than have a faster LLVM build and then a slower build of the > world ... > > The rest of this thread has determined that building LLVM accounts for > half of the build time in a clean FreeBSD build. LLVM's CMake is not a > great example: it has been incrementally improved since CMake 2.8 and > doesn't yet use any of the modern CMake features that allow > encapsulating targets and providing import / export configurations. > > In spite of that, it generates a ninja file that compiles > *significantly* faster than the bmake-based system in FreeBSD. In other > projects that I've worked on with a similar-sized codebase to FreeBSD > that use CMake + Ninja, I've never had the same problems with build > speed that I have with FreeBSD. > The speed is comparable to META_MODE, but much faster than the default. It suffers a bit because our buildworld stuff is a kinda sorts not quite good enough dependency tracker and it's layered on top of that. > Working on LLVM, I generally spend well under 10% of my time either > waiting for builds or fighting the build system. Working on FreeBSD, I > generally spend over 90% of my time waiting for builds or fighting the > build system. This means that my productivity contributing to FreeBSD > is almost zero. > > For reference, changes to LLVM typically build for me in under 30 > seconds with Ninja, unless I've changed a header that everything > > In particular, building FreeBSD on a 10-24 core machine has very long > periods where a number of the cores are completely idle. > > Ninja also has a few other nice features that improve performance > relative to bmake: > > - It lets you put jobs in different pools. In LLVM this is used to > put link and compile jobs in different pools because linking with LLD > uses multiple threads and a lot more memory than compilation, so a > 10-core machine may want to do 12 compile jobs in parallel but only 2 > link jobs. This makes it much easier to completely saturate the machine. > This is nice. > - Ninja provides each parallel build task with a separate pipe for > stdout and stderr, and does not print their output unless a build step > fails (or unless you build with -v). With bmake, if a parallel build > fails I have to rerun the build without -j, because the output is > interleaved with succeeding jobs and it's difficult to see what actually > failed. With ninja, the output is from each failed job, with no > interleaving. > META_MODE does this too. It's the primary reason I use it, along with the speed. The big downside of ninja make is that it requires you to have some other, higher-level build system to generate the Makefiles. And my experience to date with meson is a mixed bag: it's quite a bit better than normal bmake for some things, and quite a bit worse for others in terms of the code you need to write to get things done, or the efforts required to debug mistakes. I've not used cmake as the ninja makefile generator, so I have no comments on it. Doing the conversion to at least meson would be a quite large job, at least by my initial estimates. Hacking bmake to generate ninja makefiles looks to be quite a bit simpler... But looks can be deceiving. Warner --00000000000098786c05cb7b4836--