Re: git: 9cabef3d146e - main - ldd: use direct exec mode unconditionally

From: Jessica Clarke <jrtc27_at_freebsd.org>
Date: Sat, 22 Oct 2022 01:32:22 UTC
On 22 Oct 2022, at 02:13, Konstantin Belousov <kostikbel@gmail.com> wrote:
> 
> On Fri, Oct 21, 2022 at 03:10:59PM +0200, Kristof Provost wrote:
>> On 21 Oct 2022, at 15:07, Konstantin Belousov wrote:
>>> On Fri, Oct 21, 2022 at 02:54:59PM +0200, Kristof Provost wrote:
>>>> On 21 Oct 2022, at 14:50, Konstantin Belousov wrote:
>>>>> On Fri, Oct 21, 2022 at 02:18:04PM +0200, Kristof Provost wrote:
>>>>>> On 6 Oct 2022, at 17:50, Konstantin Belousov wrote:
>>>>>>> The branch main has been updated by kib:
>>>>>>> 
>>>>>>> URL: https://cgit.FreeBSD.org/src/commit/?id=9cabef3d146e9a844813b6bc8952d6cf2e9d45e5
>>>>>>> 
>>>>>>> commit 9cabef3d146e9a844813b6bc8952d6cf2e9d45e5
>>>>>>> Author: Konstantin Belousov <kib@FreeBSD.org>
>>>>>>> AuthorDate: 2022-09-21 13:55:44 +0000
>>>>>>> Commit: Konstantin Belousov <kib@FreeBSD.org>
>>>>>>> CommitDate: 2022-10-06 15:50:26 +0000
>>>>>>> 
>>>>>>> ldd: use direct exec mode unconditionally
>>>>>>> 
>>>>>>> Trying to exec malformed or unusual binary, for instance, a
>>>>>>> non-FreeBSD
>>>>>>> ABI, or using a non-standard interpreter, might give
>>>>>>> unexpected
>>>>>>> outcome.
>>>>>>> 
>>>>>>> Reported by: The UK's National Cyber Security Centre (NCSC)
>>>>>>> Reviewed by: emaste, markj, philip
>>>>>>> Discussed with: jhb
>>>>>>> Sponsored by: The FreeBSD Foundation
>>>>>>> admbug: 991
>>>>>>> PR: 127276, 175339, 231926
>>>>>>> MFC after: 1 week
>>>>>>> Differential revision: https://reviews.freebsd.org/D36650
>>>>>>> 
>>>>>> This appears to break things for armv7 (running on aarch64).
>>>>>> 
>>>>>> This manifests while building pfsense (for 3100 / armv7), which we
>>>>>> do on an
>>>>>> aarch64 vm (to avoid having to deal with qemu, and because it’s
>>>>>> faster).
>>>>>> 
>>>>>> During that build a couple ports fail to build, including
>>>>>> databases/sqlite3.
>>>>>> It fails running `/usr/bin/ldd -a "/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3" "/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/lib/libsqlite3.so”`,
>>>>>> which produces:
>>>>>> 
>>>>>> 	ld-elf.so.1: /wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
>>>>>> mmap of entire address space failed: Cannot allocate memory
>>>>>> 	/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
>>>>>> exit status 1
>>>>>> 
>>>>>> That fails doing the `mapbase = mmap(base_addr, mapsize, PROT_NONE,
>>>>>> base_flags, -1, 0);` call in rtld-elf’s map_object():217.
>>>>>> That call
>>>>>> does
>>>>>> `mmap(0x10000, 0x1dc000, PROT_NONE, 0x6010, -1, 0) => 0xffffffff`.
>>>>>> 
>>>>>> With this patch reverted we can build successfully.
>>>>> 
>>>>> Can you manually invoke ldd on the binary under ktrace -i, and
>>>>> show me
>>>>> the
>>>>> kdump output?
>>>>> 
>>>> I might be doing something wrong:
>>>> 
>>>> 	# ktrace -i /usr/obj/usr/src/arm.armv7/usr.bin/ldd/ldd -a "/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3"
>>>> 	ld-elf.so.1: /wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
>>>> mmap of entire address space failed: Cannot allocate memory
>>>> 	/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
>>>> exit status 1
>>>> 	# kdump -f ktrace.out
>>>> 	 16 @ UNKNOWN(265)
>>>> 	kdump: data too short
>>>> 	#
>>>> 
>>>> Perhaps because this is running in a jail?
>>> You need to run host-native kdump, because your kernel is arm64, I
>>> guess.
>>> 
>> That seemed to do the trick:
> 
> Please try something along this lines:
> 
> diff --git a/sys/arm64/include/elf.h b/sys/arm64/include/elf.h
> index 3f7c3964d428..22e968c632bf 100644
> --- a/sys/arm64/include/elf.h
> +++ b/sys/arm64/include/elf.h
> @@ -86,7 +86,7 @@ __ElfType(Auxinfo);
> #endif
> 
> #if __ELF_WORD_SIZE == 32
> -#define	ET_DYN_LOAD_ADDR 0x12000
> +#define	ET_DYN_LOAD_ADDR 0x01001000
> #else
> #define	ET_DYN_LOAD_ADDR 0x100000
> #endif

Ouch... sys/arm’s is better at 0x500000 but that’s still only ~4.9 MiB
above LLD’s default image base of 0x10000 so I could see that being hit
a bunch. Then again 0x01001000 only gives ~15.9 MiB, which isn’t great,
but other architectures seem bad too...

Jess