Re: git: 9cabef3d146e - main - ldd: use direct exec mode unconditionally

On Sat, Oct 22, 2022 at 02:32:22AM +0100, Jessica Clarke wrote:
> On 22 Oct 2022, at 02:13, Konstantin Belousov <kostikbel@gmail.com> wrote:
> > 
> > On Fri, Oct 21, 2022 at 03:10:59PM +0200, Kristof Provost wrote:
> >> On 21 Oct 2022, at 15:07, Konstantin Belousov wrote:
> >>> On Fri, Oct 21, 2022 at 02:54:59PM +0200, Kristof Provost wrote:
> >>>> On 21 Oct 2022, at 14:50, Konstantin Belousov wrote:
> >>>>> On Fri, Oct 21, 2022 at 02:18:04PM +0200, Kristof Provost wrote:
> >>>>>> On 6 Oct 2022, at 17:50, Konstantin Belousov wrote:
> >>>>>>> The branch main has been updated by kib:
> >>>>>>> 
> >>>>>>> URL: https://cgit.FreeBSD.org/src/commit/?id=9cabef3d146e9a844813b6bc8952d6cf2e9d45e5
> >>>>>>> 
> >>>>>>> commit 9cabef3d146e9a844813b6bc8952d6cf2e9d45e5
> >>>>>>> Author: Konstantin Belousov <kib@FreeBSD.org>
> >>>>>>> AuthorDate: 2022-09-21 13:55:44 +0000
> >>>>>>> Commit: Konstantin Belousov <kib@FreeBSD.org>
> >>>>>>> CommitDate: 2022-10-06 15:50:26 +0000
> >>>>>>> 
> >>>>>>> ldd: use direct exec mode unconditionally
> >>>>>>> 
> >>>>>>> Trying to exec malformed or unusual binary, for instance, a
> >>>>>>> non-FreeBSD
> >>>>>>> ABI, or using a non-standard interpreter, might give
> >>>>>>> unexpected
> >>>>>>> outcome.
> >>>>>>> 
> >>>>>>> Reported by: The UK's National Cyber Security Centre (NCSC)
> >>>>>>> Reviewed by: emaste, markj, philip
> >>>>>>> Discussed with: jhb
> >>>>>>> Sponsored by: The FreeBSD Foundation
> >>>>>>> admbug: 991
> >>>>>>> PR: 127276, 175339, 231926
> >>>>>>> MFC after: 1 week
> >>>>>>> Differential revision: https://reviews.freebsd.org/D36650
> >>>>>>> 
> >>>>>> This appears to break things for armv7 (running on aarch64).
> >>>>>> 
> >>>>>> This manifests while building pfsense (for 3100 / armv7), which we
> >>>>>> do on an
> >>>>>> aarch64 vm (to avoid having to deal with qemu, and because it’s
> >>>>>> faster).
> >>>>>> 
> >>>>>> During that build a couple ports fail to build, including
> >>>>>> databases/sqlite3.
> >>>>>> It fails running `/usr/bin/ldd -a "/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3" "/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/lib/libsqlite3.so”`,
> >>>>>> which produces:
> >>>>>> 
> >>>>>> 	ld-elf.so.1: /wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
> >>>>>> mmap of entire address space failed: Cannot allocate memory
> >>>>>> 	/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
> >>>>>> exit status 1
> >>>>>> 
> >>>>>> That fails doing the `mapbase = mmap(base_addr, mapsize, PROT_NONE,
> >>>>>> base_flags, -1, 0);` call in rtld-elf’s map_object():217.
> >>>>>> That call
> >>>>>> does
> >>>>>> `mmap(0x10000, 0x1dc000, PROT_NONE, 0x6010, -1, 0) => 0xffffffff`.
> >>>>>> 
> >>>>>> With this patch reverted we can build successfully.
> >>>>> 
> >>>>> Can you manually invoke ldd on the binary under ktrace -i, and
> >>>>> show me
> >>>>> the
> >>>>> kdump output?
> >>>>> 
> >>>> I might be doing something wrong:
> >>>> 
> >>>> 	# ktrace -i /usr/obj/usr/src/arm.armv7/usr.bin/ldd/ldd -a "/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3"
> >>>> 	ld-elf.so.1: /wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
> >>>> mmap of entire address space failed: Cannot allocate memory
> >>>> 	/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
> >>>> exit status 1
> >>>> 	# kdump -f ktrace.out
> >>>> 	 16 @ UNKNOWN(265)
> >>>> 	kdump: data too short
> >>>> 	#
> >>>> 
> >>>> Perhaps because this is running in a jail?
> >>> You need to run host-native kdump, because your kernel is arm64, I
> >>> guess.
> >>> 
> >> That seemed to do the trick:
> > 
> > Please try something along this lines:
> > 
> > diff --git a/sys/arm64/include/elf.h b/sys/arm64/include/elf.h
> > index 3f7c3964d428..22e968c632bf 100644
> > --- a/sys/arm64/include/elf.h
> > +++ b/sys/arm64/include/elf.h
> > @@ -86,7 +86,7 @@ __ElfType(Auxinfo);
> > #endif
> > 
> > #if __ELF_WORD_SIZE == 32
> > -#define	ET_DYN_LOAD_ADDR 0x12000
> > +#define	ET_DYN_LOAD_ADDR 0x01001000
> > #else
> > #define	ET_DYN_LOAD_ADDR 0x100000
> > #endif
> 
> Ouch... sys/arm’s is better at 0x500000 but that’s still only ~4.9 MiB
> above LLD’s default image base of 0x10000 so I could see that being hit
> a bunch. Then again 0x01001000 only gives ~15.9 MiB, which isn’t great,
> but other architectures seem bad too...

I put initial tweaking change into https://reviews.freebsd.org/D37085
Feel free to comment/suggest better load addresses for use.