Cross-building with -fsanitize [was Re: Brokenness in i386 cross-build]

[Bryan Drewery]

Using -fsanitize with WITH_SYSTEM_COMPILER (meaning a --sysroot and not
a default built-in path and using /usr/bin/cc), or an external compiler,
lacks proper support for building of all target cross-archs.

There's some jumbled back and forth context below my mail about it as well.

Normally without WITH_SYSTEM_COMPILER clang will compute the path to the
libusan*.a files by the path with which it was executed.  This is for
the libclang_rt lookups.  That is normally going to be
WORLDTMP/usr/bin/../lib/...  Don't forget that WORLDTMP is our sysroot.

When we use WITH_SYSTEM_COMPILER it will build with /usr/bin/cc rather
than the WORLDTMP one.  Thus it looks for the libusan support on the host:

>> /usr/bin/../lib/clang/3.8.0/lib/freebsd/libclang_rt.ubsan_standalone-i386.a

We do actually build the proper libclang_rt/libusan files for the
cross-build and install them into the sysroot.

This path comes from CompilerInvocation::GetResourcesPath() which lacks
a run-time override or --sysroot or -B support.

We only ever build the native libusan files and if WITH_LIB32 is
supported and enabled then you also get the i386.a files.

If you build i386 from amd64 WITH_SYSTEM_COMPILER(default) and the host
has WITHOUT_LIB32, then you will not have the
/usr/lib/clang/3.8.0/lib/freebsd/libclang_rt.ubsan_standalone-i386.a
file that is found by the running /usr/bin/cc when trying to use
-fsanitize and -target=i386.

I'm not sure if it is proper for clang to use its internal versions here
or perhaps use the sysrooted versions.  The ResourcesPath seems to also
affect the internal header paths for builtin includes.  So we should be
cautious with it.  As Xin Li pointed out we could possibly add the
sysroot in only for the libusan/libclang_rt lookups.

If clang should only use its own versions of these .a files even with a
--sysroot then we need to build the libusan*.a files for every arch that
we support for the clang that is installed.  Just as we build -target
support for every version of clang that we support for the clang that is
installed.

Hope this all made sense.

As for the build breakage, there is only 1 thing (a test) in the tree
currently using -fsanitize.  So we can just workaround it for now by
disabling cross-build for it.

More context follows.

On 6/27/16 12:30 AM, Xin Li wrote:
> Could you please modify the patch to add a comment of XXX/TODO that this
> is a workaround? (Adding Dmitry; I think this shall be solved at
> toolchain level.  I'm not totally against applying the workaround for
> now though.)
> 
> ===
> 
> It seems that in
> collectSanitizerRuntimes(contrib/llvm/tools/clang/lib/Driver/Tools.cpp:2826
> then in 2852, then eventually 2756) we prefixed the library names with
> hardcoded path, derived from getDriver().ResourceDir, which in turn
> would give /usr/bin/.. without considering sysroot setting.
> 
> So in effect we are passing something like:
> 
> -whole-archive
> /usr/bin/../lib/clang/3.8.0/lib/freebsd/libclang_rt.ubsan_standalone-i386.a
> -no-whole-archive
> 
> To ld(1), while we really need to pass:
> 
> -whole-archive
> ${sysroot}/usr/bin/../lib/clang/3.8.0/lib/freebsd/libclang_rt.ubsan_standalone-i386.a
> -no-whole-archive
> 
> Which I'm not familiar enough with the compiler internals to comfortably
> tell if we would break something else by blindly prefixing the result
> from ResourceDir.
> 
> Cheers,
> 
>> > In message <20160626215850.GB79880 at FreeBSD.org>, Glen Barber writes:
>>> >>
>>> >>
>>> >> --A6N2fC+uXW/VQSAv
>>> >> Content-Type: text/plain; charset=us-ascii
>>> >> Content-Disposition: inline
>>> >> Content-Transfer-Encoding: quoted-printable
>>> >>
>>> >> Confirmed, the build fails.
>>> >>
>>> >>
>>> >> /usr/src/contrib/netbsd-tests/lib/libc/ssp/h_gets.c:40: warning: warning: t=
>>> >> his program uses gets(), which is unsafe.
>>> >> t_mktemp.o: In function `atfu_mktemp_not_exist_body':
>>> >> /usr/src/contrib/netbsd-tests/lib/libc/stdio/t_mktemp.c:47: warning: warnin=
>>> >> g: mktemp() possibly used unsafely; consider using mkstemp()
>>> >> /usr/obj/i386.i386/usr/src/tmp/usr/bin/ld:
>>> /usr/bin/../lib/clang/3.8.0/lib/=
>>> >> freebsd/libclang_rt.ubsan_standalone-i386.a: No such file: No such file or =
>>> >> directory
>>> >> cc: error: linker command failed with exit code 1 (use -v to see invocation)
>>> >> --- h_raw.full ---
>>> >> *** [h_raw.full] Error code 1
>>> >>
>>> >> make[7]: stopped in /usr/src/lib/libc/tests/ssp



-- 
Regards,
Bryan Drewery

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-toolchain/attachments/20160627/e09b7b11/attachment.sig>