Re: git: 5e6a2d6eb220 - main - Reapply: move libc++ from /usr/lib to /lib [add /usr/lib/libc++.so.1 -> ../../lib/libc++.so.1 ?]

On 2021-Dec-31, at 17:46, Mark Millard <marklmi@yahoo.com> wrote:

> On 2021-Dec-31, at 15:04, John Baldwin <jhb@freebsd.org> wrote:
> 
>> On 12/31/21 2:59 PM, Mark Millard wrote:
>>> On 2021-Dec-31, at 14:28, Mark Millard <marklmi@yahoo.com> wrote:
>>>> On 2021-Dec-30, at 14:04, John Baldwin <jhb@freebsd.org> wrote:
>>>> 
>>>>> On 12/30/21 1:09 PM, Mark Millard wrote:
>>>>>> On 2021-Dec-30, at 13:05, Mark Millard <marklmi@yahoo.com> wrote:
>>>>>>> This asks a question in a different direction that my prior
>>>>>>> reports about my builds vs. Cy's reported build.
>>>>>>> 
>>>>>>> Background:
>>>>>>> 
>>>>>>> /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/usr/lib/libc++.so:GROUP ( /lib/libc++.so.1 /usr/lib/libcxxrt.so
>>>>>>> and:
>>>>>>> lrwxr-xr-x  1 root  wheel        23 Dec 29 13:17:01 2021 /usr/lib/libcxxrt.so -> ../../lib/libcxxrt.so.1
>>>>>>> 
>>>>>>> Why did libc++.so.1 not get a:
>>>>>>> 
>>>>>>> /usr/lib/libc++.so.1 -> ../../lib/libc++.so.1
>>>>>> I forgot to remove the .1 on the left hand side:
>>>>>> /usr/lib/libc++.so -> ../../lib/libc++.so.1
>>>>> 
>>>>> Because for libc++.so we don't just symlink to the current version of the library
>>>>> (as we do for most other shared libraries) to tell the compiler what to link against
>>>>> for -lc++, instead we use a linker script that tells the compiler to link against
>>>>> both of those libraries when -lc++ is encountered.
>>>> 
>>>> A better identification of what looks odd to me is the
>>>> path variations in:
>>>> 
>>>> # more /usr/lib/libc++.so
>>> Another not great day on my part: That path alone makes
>>> the mix of /lib/ and /usr/lib/ use involved, given the
>>> reference to /lib/libc++.so.1 . That would still be true
>>> if the other path had been /lib/libcxxrt.so .
>> 
>> /usr/lib/libc++.so is only used by the compiler/linker when linking a binary.
>> The resulting binary has the associated paths (/lib/libc++.so.1 and
>> /usr/lib/libcxxrt.so.1) in its DT_NEEDED.  So it is fine for the .so to be
>> in /usr/lib.  This is the same with /usr/lib/libc.so vs /lib/libc.so.7.
>> 
>> However, your point about libcxxrt.so.1 is valid.  It needs to also be moved
>> to /lib if libc++.so.1 is moved to /lib.  Doing so will also require yet another
>> depend-clean.sh fixup (well, probably just adjusting the one I added to
>> check the libcxxrt path instead of libc++ path).
> 
> Hmm. Looking (now after having updated so /lib/libc++.so.1
> is in use, not that this is any different here):
> 
> # ls -Tld /lib/libcxx* /usr/lib/libcxx*
> -r--r--r--  1 root  wheel  131656 Dec 31 14:19:49 2021 /lib/libcxxrt.so.1
> -r--r--r--  1 root  wheel  355764 Dec 24 15:19:42 2021 /usr/lib/libcxxrt.a
> lrwxr-xr-x  1 root  wheel      23 Dec 31 14:19:49 2021 /usr/lib/libcxxrt.so -> ../../lib/libcxxrt.so.1
> 
> # more /usr/lib/libc++.so 
> /* $FreeBSD$ */
> GROUP ( /lib/libc++.so.1 /usr/lib/libcxxrt.so )
> 
> So: no actual reference to /usr/lib/libcxxrt.so.1 but
> a reference in the DT_NEEDED to /usr/lib/libcxxrt.so ?
> 
> May be just /usr/lib/libc++.so needs different text in order
> for DT_NEEDED to have different text related to libcxxrt in
> future build activities, avoiding /usr/lib/ ?
> 
> 
> For reference:
> 
> # uname -apKU
> FreeBSD amd64_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #27 main-n252090-5650d340ad66-dirty: Fri Dec 31 06:00:41 PST 2021     root@amd64_ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG  amd64 amd64 1400046 1400046
> 

In a aarch64 context I looked at an old executable via ldd -a :

# ldd -a bt
/usr/home/root/c_tests/bt:
	libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x41c19000)
	libc++.so.1 => /lib/libc++.so.1 (0x42484000)
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x43038000)
	libm.so.5 => /lib/libm.so.5 (0x44a4c000)
	libc.so.7 => /lib/libc.so.7 (0x439ce000)
/usr/lib/libexecinfo.so.1:
	libelf.so.2 => /lib/libelf.so.2 (0x4581e000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x46e4f000)
	libc.so.7 => /lib/libc.so.7 (0x439ce000)
/lib/libc++.so.1:
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x43038000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x46e4f000)
	libc.so.7 => /lib/libc.so.7 (0x439ce000)
/lib/libcxxrt.so.1:
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x46e4f000)
	libc.so.7 => /lib/libc.so.7 (0x439ce000)
/lib/libm.so.5:
	libc.so.7 => /lib/libc.so.7 (0x439ce000)
/lib/libelf.so.2:
	libc.so.7 => /lib/libc.so.7 (0x439ce000)
/lib/libgcc_s.so.1:
	libc.so.7 => /lib/libc.so.7 (0x439ce000)

Looks like something already deals with finding
/lib/libcxxrt.so.1 . But it is not obvious what
path it started with and how much processing was
done (or when) to end up with /lib/libc++.so.1
showing.

But there is still a /usr/lib/ reference overall:

/usr/lib/libexecinfo.so.1

But this is because the old program turned out to
be an old experiment:

# more bt.c 
// bt.c
// from releng/12 (12.2?) context (pre-llvm12), but not releng/13 :
// # cc -o bt bt.c -lexecinfo
// # ./bt
// Rerun in llvm12 context, such as main after the switch: crash.

#include <execinfo.h>
int main() {
       void *addrlist[100];
       backtrace(addrlist, 100);
}

Although, for some reason, the executable was dated
2021-Jul-15, not that I remember why I'd rebuilt it
then.

# file bt
bt: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.0 (1400025), FreeBSD-style, with debug_info, not stripped

nm -a bt shows:

entry: 0
        d_tag: DT_NEEDED
        d_val: libexecinfo.so.1

None of the DT_NEEDED entries had paths shown.

The only /usr/lib/ reference was:

entry: 65
        st_name: /usr/lib/crti.o
        st_value: 0
        st_size: 0
        st_info: STT_FILE STB_LOCAL
        st_shndx: 65521

Overall: backtrace requires /usr/lib/ accessibility
for main-n252090-5650d340ad66 in order to access
/usr/lib/libexecinfo.so.1 .

For reference:

# uname -apKU
FreeBSD CA72_16Gp_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #34 main-n252090-5650d340ad66-dirty: Fri Dec 31 06:30:22 PST 2021     root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72  arm64 aarch64 1400046 1400046


===
Mark Millard
marklmi at yahoo.com