From nobody Mon Aug 02 13:11:32 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B61E012DCFA9 for ; Mon, 2 Aug 2021 13:11:41 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com [IPv6:2607:f8b0:4864:20::f2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gddgb3RTNz3tgJ for ; Mon, 2 Aug 2021 13:11:39 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qv1-xf2d.google.com with SMTP id js7so6252484qvb.4 for ; Mon, 02 Aug 2021 06:11:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=fb5ighB+7gbPjDOsd5/opKfAfsp/MvPqJvno7Kz6tyU=; b=KOKENdrzFzRpHIHqgErvcNgIZeFdTcO3/Gi/d38N/9M2oEyjoPXK/lg+G8Z4Sikudg ekFuzmT8ZO1sxoH0nflr0OmpW/Ul3N15YTXdRnDLVG4Os4UuN+GcfuLqe3cDaYhAxUsG vKj/f50Lggrbmv4Oqe9x38GmaopD4791qpxzxl6bIPcKmNm5g1sbV69HTMGcEeKtLi6J 4J8x/BNDxpaL5Cywmr0baQfOTbtATdtwK+LXxjCFRuHcJ4fHX6ieSipR+xDouZUrvydy WH9BVb5Acwd07+4iUSSFoyX148Jj5S6XvwOMAO9Rv+f6isaYAXeY64H0ty+wYOOkUe/S CYsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=fb5ighB+7gbPjDOsd5/opKfAfsp/MvPqJvno7Kz6tyU=; b=m5eeNpkM+0Z8nYaVyDLHu0g2bGCJufVZxLwqJpoqE3kn3Yu53jpiPfADpa3SV/k2oh ZOvP/YAFPx1/lf4EonGk6A219mXYEF5RBlslM9Hk1cksS8jpKNqUCmh6qCDhyaFF59dL fmYHw50cO+24idvv8adL5/KVZadOg0x291IDUbngaIdPEoNXdUkRI9v9IqmdMguW8U4I T5/hihAUZkgblWYJh4aNPfO5eDdD1UukD5lEde0ve7AV8qNYnxILVDUxnHpjzZBurkzZ eGAzaE14BtBPatHQT6VtlM0fav22zq0QFxrLYIcGBsX/69O7dhoxKj4JeCdViloeAPeX K6eg== X-Gm-Message-State: AOAM531pYbhIrCWhVwaMuIfFleIAAUdM40O3sdbjtZaLbdy835ZQiYS1 hrzf97kVNiVI/CZWFrA8BnlJRQ== X-Google-Smtp-Source: ABdhPJxZVq1iDtoUC5RgDrkYJx0F8TpbaU8GWXVU+60KgsrbWvMKHTDkPwp0KR2bj9rRAcfDp/OtGw== X-Received: by 2002:a0c:ed21:: with SMTP id u1mr16266106qvq.6.1627909893682; Mon, 02 Aug 2021 06:11:33 -0700 (PDT) Received: from mutt-hbsd (pool-100-16-224-136.bltmmd.fios.verizon.net. [100.16.224.136]) by smtp.gmail.com with ESMTPSA id 18sm6047769qkv.118.2021.08.02.06.11.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 06:11:33 -0700 (PDT) Date: Mon, 2 Aug 2021 09:11:32 -0400 From: Shawn Webb To: Ed Maste Cc: "freebsd-toolchain@FreeBSD.org" , FreeBSD Hackers , Dimitry Andric , Alexander Richardson Subject: Re: Migrating to LLVM binutils tools (ar, nm, addr2line, etc.) Message-ID: <20210802131132.c7egr6cphq322qcj@mutt-hbsd> X-Operating-System: FreeBSD mutt-hbsd 14.0-CURRENT-HBSD FreeBSD 14.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="k2ouhyaedg3r5tdd" Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4Gddgb3RTNz3tgJ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=KOKENdrz; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::f2d as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org X-Spamd-Result: default: False [-5.10 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[hardenedbsd.org]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2d:from]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: N --k2ouhyaedg3r5tdd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 05, 2021 at 11:09:18AM -0400, Ed Maste wrote: > FreeBSD migrated from GNU binutils to versions from ELF Tool Chain, > starting in 2014. At that time there were no usable LLVM versions of > those tools, but they have been developing rapidly since then. Now I > think it may be prudent to migrate to the LLVM tools where they exist, > for both functionality and maintainability reasons. >=20 > I'd like to allow use of link-time optimization (LTO) in the FreeBSD > base system. LTO runs optimization passes over the entire executable > (or library) at link time and thus allows for more effective > optimization than when performed on individual compilation units. >=20 > When using LTO object files (.o) including those contained in static > library archives (.a) contain LLVM IR bitcode rather than target > object code. This means that utilities that operate on object files > need to support LLVM IR; we currently use a number of bespoke tools > and ones obtained from ELF Tool Chain that do not have this support. >=20 > Alex Richardson also pointed out that asan (address sanitizer) > produces a useful backtrace only if addr2line is llvm-symbolizer. >=20 > Like ELF Tool Chain the LLVM tools aim for command line and output > format compatibility with GNU binutils, although there are a few minor > differences. Where these cause a material issue (breaking a port or > eliminating required functionality) we can submit LLVM bugs and work > on patches. >=20 > In the past we provided build knobs to control individual utilities > (e.g. WITH_LLD_IS_LD); I'd like to avoid perpetuating that here. It > seems individual knobs (WITH_LLVM_AR_IS_AR, WITH_LLVM_NM_IS_NM, > WITH_LLVM_SYMBOLIZER_IS_ADDR2LINE etc.) will introduce extra > complexity without adding much value. >=20 > Alex is working on a patch now and will follow up shortly, but I > wanted to email the list as a heads-up, and see if there are any > comments or concerns. >=20 > Potential next steps are: > - Introduce new build knob > - Iterate on exp-runs and call for testing > - Switch to LLVM tools by default > - Major release (14.0) > - Retire knob, leaving only the LLVM implementation. Hey Ed, As background for anyone curious, HardenedBSD switched to using llvm-ar, llvm-nm, and llvm-objdump by default years ago as part of the work to start integrating Cross-DSO CFI. We've noticed one small, but important, issue with llvm-ar (which is also the same underlying program as llvm-ranlib) in some behavior that doesn't match ELF Toolchain's ar/ranlib (which I'll call elftc-ar). For most cases, when elftc-ar fails, it does not set the exitcode to non-zero. This tricks the ports tree to continue to build a port where elftc-ar actually errored. llvm-ar does the right thing in exiting with a non-zero exit code on error. However, due to this discrepency in behavior, certain ports that cause an error condition when calling ar/ranlib continue to build when elftc-ar is used, but fail to build when llvm-ar is used. I'm thinking that I'll report this same issue to the ELF Toolchain folks since elftc-ar really should exit with a non-zero exitcode on failure. I've just now hacked llvm-ar to behave the same as elftc-ar[0] and will do a poudriere bulk run soon. I'll report back my status with the ELF Toolchain notification and the poudriere run as soon as I have more info. [0]: https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/commit/5bdcc54a23f058= 83f55e895da49726955fa8b07b Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --k2ouhyaedg3r5tdd Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmEH7wIACgkQ/y5nonf4 4fqODQ/8Dp5G6Yjk861k59c2MSLnj2As3Mu7XgTe3/HPznAUg8GRgtTzwZHi52R3 lTPtHHiCtKytGr4UQKfSpdJ2NXvTJ3IWY/Ul77BlMGb3+gGVli2suf6xgkI82ZWC jsTfMyD6FlRAHc17AbuWHaCYiuarJI+CLPnA6Bcarvz+fAE1TVsqgWDcV5WtBjSD XGTZWSvDHGZmvH7evsoXkBXq1mT8C6L7LbcHecNdEYONkTnMDHL0KDGoyZovX9Zv gigqPuPHJXd6h/0xNkQ6XSmv0g0V2o0sSC8gVYN/hbDJ+oTja04r7b+3pc8R/Lw/ oVZ56SuO52Q45hsUnhxNtXfbMcDfG39ExVRfy2NRHUGu5Vh8fy3vCXv7y0rdQnX+ jX8d5k+s9s48qjzKKjWkKgdoyyBuOLq5RELp98fZs3FJem6wfMD96avdwxcgD75v uMyGaAxoU4380n0Mdlq3VwJxrP9uPVhb0kVHCGs088i1d6lOP4cXjaEZxHxdW+wz awm6PRHYcofx4JkGru52WcavIxHOWLnCwU0rHSikLlNMwU+4gv3HvD9dCh6+E0+C DHyoOS7UxaFnNYEujCZz9PjRL8CpCfZV5vzfxcr2m1V1wxYHCRZnOcN9vV3BJ4gM rAG/RG66Vln6PUv1kywqby9Iv6FdR7plc3O9hRAr6rGFKp3RLyc= =jthh -----END PGP SIGNATURE----- --k2ouhyaedg3r5tdd--