Migrating to LLVM binutils tools (ar, nm, addr2line, etc.)

From: Ed Maste <emaste_at_freebsd.org>
Date: Mon, 05 Jul 2021 15:09:18 UTC
FreeBSD migrated from GNU binutils to versions from ELF Tool Chain,
starting in 2014. At that time there were no usable LLVM versions of
those tools, but they have been developing rapidly since then. Now I
think it may be prudent to migrate to the LLVM tools where they exist,
for both functionality and maintainability reasons.

I'd like to allow use of link-time optimization (LTO) in the FreeBSD
base system. LTO runs optimization passes over the entire executable
(or library) at link time and thus allows for more effective
optimization than when performed on individual compilation units.

When using LTO object files (.o) including those contained in static
library archives (.a) contain LLVM IR bitcode rather than target
object code. This means that utilities that operate on object files
need to support LLVM IR; we currently use a number of bespoke tools
and ones obtained from ELF Tool Chain that do not have this support.

Alex Richardson also pointed out that asan (address sanitizer)
produces a useful backtrace only if addr2line is llvm-symbolizer.

Like ELF Tool Chain the LLVM tools aim for command line and output
format compatibility with GNU binutils, although there are a few minor
differences. Where these cause a material issue (breaking a port or
eliminating required functionality) we can submit LLVM bugs and work
on patches.

In the past we provided build knobs to control individual utilities
(e.g. WITH_LLD_IS_LD); I'd like to avoid perpetuating that here. It
seems individual knobs (WITH_LLVM_AR_IS_AR, WITH_LLVM_NM_IS_NM,
WITH_LLVM_SYMBOLIZER_IS_ADDR2LINE etc.) will introduce extra
complexity without adding much value.

Alex is working on a patch now and will follow up shortly, but I
wanted to email the list as a heads-up, and see if there are any
comments or concerns.

Potential next steps are:
- Introduce new build knob
- Iterate on exp-runs and call for testing
- Switch to LLVM tools by default
- Major release (14.0)
- Retire knob, leaving only the LLVM implementation.