Re: performance regressions in 15.0

Konstantin Belousov <kib_at_freebsd.org> wrote on
Date: Sat, 06 Dec 2025 17:42:57 UTC :

> On Sat, Dec 06, 2025 at 06:34:01PM +0100, Mateusz Guzik wrote:
> > On Sat, Dec 6, 2025 at 6:31 PM Mateusz Guzik <mjguzik@gmail.com> wrote:
> > >
> > > On Sat, Dec 6, 2025 at 6:25 PM Konstantin Belousov <kib@freebsd.org> wrote:
> > > >
> > > > On Sat, Dec 06, 2025 at 11:50:08AM +0100, Mateusz Guzik wrote:
> > > > > II. compilation speed
> > > > >
> > > > > The the real and serious problem. Both versions of the system ship the
> > > > > same clang version:
> > > > > FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git
> > > > > llvmorg-19.1.7-0-gcd708029e0b2)
> > > > > Target: x86_64-unknown-freebsd14.3
> > > > > Thread model: posix
> > > > > InstalledDir: /usr/bin
> > > > >
> > > > > FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git
> > > > > llvmorg-19.1.7-0-gcd708029e0b2)
> > > > > Target: x86_64-unknown-freebsd15.0
> > > > > Thread model: posix
> > > > > InstalledDir: /usr/bin
> > > > >
> > > > > I found that compiling the will-it-scale suite about doubles in real
> > > > > time needed, along with doubling time spent in userspace.
> > > > >
> > > > > will-it-scale needs a little bit of massaging to work, diff at the end.
> > > > >
> > > > > check this out (repeabale): while true; do gmake -s clean && time
> > > > > gmake -s -j 8; done
> > > > >
> > > > > 14.3:
> > > > > gmake -s -j 8 8.93s user 2.03s system 769% cpu 1.42s (1.424) total
> > > > > gmake -s -j 8 9.02s user 2.16s system 757% cpu 1.48s (1.475) total
> > > > > gmake -s -j 8 9.29s user 1.95s system 774% cpu 1.45s (1.450) total
> > > > > gmake -s -j 8 8.97s user 2.46s system 770% cpu 1.48s (1.484) total
> > > > > gmake -s -j 8 9.13s user 2.30s system 773% cpu 1.48s (1.477) total
> > > > 14.3 clang/lld are probably statically linked, but I am not sure.
> > > > Can you confirm this?
> > > >
> > >
> > > They are dynamic, this was switched years ago.
> > >
> > > /usr/bin/clang: ELF 64-bit LSB executable, x86-64, version 1
> > > (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for
> > > FreeBSD 14.3, FreeBSD-style, stripped
> > >
> > 
> > ... but I do see a change in used .sos:
> > 
> > /usr/bin/cc:
> > libz.so.6 => /lib/libz.so.6 (0x3d76fc831000)
> > libprivatezstd.so.5 => /usr/lib/libprivatezstd.so.5 (0x3d76fcc0e000)
> > libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x3d76fe213000)
> > libncursesw.so.9 => /lib/libncursesw.so.9 (0x3d76fd1ef000)
> > libthr.so.3 => /lib/libthr.so.3 (0x3d76ff8e5000)
> > libc++.so.1 => /lib/libc++.so.1 (0x3d76ff002000)
> > libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x3d770073c000)
> > libm.so.5 => /lib/libm.so.5 (0x3d7701d4c000)
> > libc.so.7 => /lib/libc.so.7 (0x3d770161d000)
> > libelf.so.2 => /lib/libelf.so.2 (0x3d7701e58000)
> > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x3d770314a000)
> > libtinfow.so.9 => /lib/libtinfow.so.9 (0x3d7702808000)
> > [vdso] (0x3d76fc3cc000)
> > 
> > vs
> > 
> > /usr/bin/cc:
> > libprivateclang.so.19 => /usr/lib/libprivateclang.so.19 (0x311e43c00000)
> > libprivatellvm.so.19 => /usr/lib/libprivatellvm.so.19 (0x311e48000000)
> > libz.so.6 => /lib/libz.so.6 (0x311e4d1a9000)
> > libprivatezstd.so.5 => /usr/lib/libprivatezstd.so.5 (0x311e430cf000)
> > libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x311e4e075000)
> > libncursesw.so.9 => /lib/libncursesw.so.9 (0x311e4e7b3000)
> > libthr.so.3 => /lib/libthr.so.3 (0x311e4f717000)
> > libc++.so.1 => /lib/libc++.so.1 (0x311e502e0000)
> > libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x311e5107d000)
> > libm.so.5 => /lib/libm.so.5 (0x311e52294000)
> > libc.so.7 => /lib/libc.so.7 (0x311e532a7000)
> > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x311e5414b000)
> > libelf.so.2 => /lib/libelf.so.2 (0x311e514d0000)
> > libtinfow.so.9 => /lib/libtinfow.so.9 (0x311e54751000)
> > libsys.so.7 => /lib/libsys.so.7 (0x311e563ff000)
> > [vdso] (0x311e419d5000)
> > 
> > as in the 14 binary does not link to libprivateclang nor libprivatellm
> Well, yes, I mean, the internal llvm libc were linked statically in
> 14.3. This is where the count of relocations I posted comes from.

FYI:

JAILNAME                   VERSION      OSVERSION ARCH  METHOD    TIMESTAMP           PATH
official14-i386            14.3-STABLE  1403506   i386  freebsdci 2025-11-28 15:52:21 /usr/local/poudriere/jails/official14-i386

ldd /usr/local/poudriere/jails/official14-i386/usr/bin/cc | grep so.19
	libprivateclang.so.19 => not found (0)
	libprivatellvm.so.19 => not found (0)

So the change was MFC'd to stable/14 .

> > 
> > 
> > 
> > > > >
> > > > > 15.0:
> > > > > gmake -s -j 8 19.90s user 3.02s system 773% cpu 2.96s (2.963) total
> > > > > gmake -s -j 8 19.90s user 3.18s system 774% cpu 2.98s (2.979) total
> > > > > gmake -s -j 8 20.24s user 2.90s system 770% cpu 3.00s (3.005) total
> > > > > gmake -s -j 8 19.92s user 3.25s system 771% cpu 3.00s (3.003) total
> > > > > gmake -s -j 8 20.25s user 2.95s system 772% cpu 3.01s (3.006) total
> > > > But 15.0 is definitely dynamically linked.
> > > >
> > > > clang is enormous C++ binary with enormous amount of relocs:
> > > > $ ldd /usr/bin/cc
> > > > /usr/bin/cc:
> > > > libprivateclang.so.19 => /usr/lib/libprivateclang.so.19 (0x27417e200000)
> > > > libprivatellvm.so.19 => /usr/lib/libprivatellvm.so.19 (0x274183e00000)
> > > >
> > > > $ objdump -R /usr/lib/libprivateclang.so.19 | wc -l
> > > > 232977
> > > > $ objdump -R /usr/lib/libprivatellvm.so.19 | wc -l
> > > > 140712
> > > >
> > > >
> > > > >
> > > > > user time *skyrocketed*
> > > >



===
Mark Millard
marklmi at yahoo.com