Re: -CURRENT compilation time

On Wed, 8 Sep 2021 09:57:50 +0100
David Chisnall <theraven@FreeBSD.org> wrote:

> On 07/09/2021 18:02, Stefan Esser wrote:
> > Wouldn't this break META_MODE?  
> 
> I have never managed to get META_MODE to work but my understanding
> is that META_MODE is addressing a problem that doesn't really exist
> in any other build system that I've used:  that dependencies are not
> properly tracked.
> 

META_MODE requires filemon(4) to be in the kernel or loaded as a module.
make(1) will use if it's available with .MAKE.MODE=meta.

> When I do a build of LLVM with the upstream build system with no
> changes, it takes Ninja approximately a tenth of a second to stat
> all of the relevant files and tell me that I have no work to do. 
> META_MODE apparently lets the FreeBSD build system extract these
> dependencies and do something similar, but it's not enabled by
> default and it's difficult to make work.
> 
> > I'd rather be able to continue building the world within a few
> > minutes (generally much less than 10 minutes, as long as there is
> > no major LLVM upgrade) than have a faster LLVM build and then a
> > slower build of the world ... 
> 
> The rest of this thread has determined that building LLVM accounts
> for half of the build time in a clean FreeBSD build.  LLVM's CMake
> is not a great example:  it has been incrementally improved since
> CMake 2.8 and doesn't yet use any of the modern CMake features that
> allow encapsulating targets and providing import / export
> configurations.
> 
> In spite of that, it generates a ninja file that compiles
> *significantly* faster than the bmake-based system in FreeBSD.  In
> other projects that I've worked on with a similar-sized codebase to
> FreeBSD that use CMake + Ninja, I've never had the same problems
> with build speed that I have with FreeBSD.
> 
> Working on LLVM, I generally spend well under 10% of my time either
> waiting for builds or fighting the build system.  Working on
> FreeBSD, I generally spend over 90% of my time waiting for builds or
> fighting the build system.  This means that my productivity
> contributing to FreeBSD is almost zero.
> 
> For reference, changes to LLVM typically build for me in under 30
> seconds with Ninja, unless I've changed a header that everything
> 
> In particular, building FreeBSD on a 10-24 core machine has very
> long periods where a number of the cores are completely idle.
> 
> Ninja also has a few other nice features that improve performance
> relative to bmake:
> 
> - It lets you put jobs in different pools.  In LLVM this is used to
> put link and compile jobs in different pools because linking with
> LLD uses multiple threads and a lot more memory than compilation, so
> a 10-core machine may want to do 12 compile jobs in parallel but
> only 2 link jobs.  This makes it much easier to completely saturate
> the machine.
> - Ninja provides each parallel build task with a separate pipe for
> stdout and stderr, and does not print their output unless a build
> step fails (or unless you build with -v).  With bmake, if a parallel
> build fails I have to rerun the build without -j, because the output
> is interleaved with succeeding jobs and it's difficult to see what
> actually failed.  With ninja, the output is from each failed job,
> with no interleaving.
> 

ninja sounds really neat and it's available as /usr/ports/devel/ninja.

Seems to me that there was an earlier mail about getting CMAKE to work
with FreeBSD builds.  Could be worthwhile to look into getting ninja
to work also.  But I could understand that there might be push-back,
since the project prefers to use utilities from the source tree.

-- 
Gary Jennejohn