Re: git: 9cabef3d146e - main - ldd: use direct exec mode unconditionally

From: Kristof Provost <kp_at_FreeBSD.org>
Date: Fri, 21 Oct 2022 12:54:59 UTC
On 21 Oct 2022, at 14:50, Konstantin Belousov wrote:
> On Fri, Oct 21, 2022 at 02:18:04PM +0200, Kristof Provost wrote:
>> On 6 Oct 2022, at 17:50, Konstantin Belousov wrote:
>>> The branch main has been updated by kib:
>>>
>>> URL: 
>>> https://cgit.FreeBSD.org/src/commit/?id=9cabef3d146e9a844813b6bc8952d6cf2e9d45e5
>>>
>>> commit 9cabef3d146e9a844813b6bc8952d6cf2e9d45e5
>>> Author:     Konstantin Belousov <kib@FreeBSD.org>
>>> AuthorDate: 2022-09-21 13:55:44 +0000
>>> Commit:     Konstantin Belousov <kib@FreeBSD.org>
>>> CommitDate: 2022-10-06 15:50:26 +0000
>>>
>>>     ldd: use direct exec mode unconditionally
>>>
>>>     Trying to exec malformed or unusual binary, for instance, a
>>> non-FreeBSD
>>>     ABI, or using a non-standard interpreter, might give unexpected
>>> outcome.
>>>
>>>     Reported by:    The UK's National Cyber Security Centre (NCSC)
>>>     Reviewed by:    emaste, markj, philip
>>>     Discussed with: jhb
>>>     Sponsored by:   The FreeBSD Foundation
>>>     admbug: 991
>>>     PR:     127276, 175339, 231926
>>>     MFC after:      1 week
>>>     Differential revision:  https://reviews.freebsd.org/D36650
>>>
>> This appears to break things for armv7 (running on aarch64).
>>
>> This manifests while building pfsense (for 3100 / armv7), which we do 
>> on an
>> aarch64 vm (to avoid having to deal with qemu, and because it’s 
>> faster).
>>
>> During that build a couple ports fail to build, including 
>> databases/sqlite3.
>> It fails running `/usr/bin/ldd -a 
>> "/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3" 
>> "/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/lib/libsqlite3.so”`,
>> which produces:
>>
>> 	ld-elf.so.1: 
>> /wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
>> mmap of entire address space failed: Cannot allocate memory
>> 	/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3:
>> exit status 1
>>
>> That fails doing the `mapbase = mmap(base_addr, mapsize, PROT_NONE,
>> base_flags, -1, 0);` call in rtld-elf’s map_object():217. That call 
>> does
>> `mmap(0x10000, 0x1dc000, PROT_NONE, 0x6010, -1, 0) => 0xffffffff`.
>>
>> With this patch reverted we can build successfully.
>
> Can you manually invoke ldd on the binary under ktrace -i, and show me 
> the
> kdump output?
>
I might be doing something wrong:

	# ktrace -i /usr/obj/usr/src/arm.armv7/usr.bin/ldd/ldd -a 
"/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3"
	ld-elf.so.1: 
/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3: 
mmap of entire address space failed: Cannot allocate memory
	/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3: 
exit status 1
	# kdump -f ktrace.out
	    16 @      UNKNOWN(265)
	kdump: data too short
	#

Perhaps because this is running in a jail?

Here’s truss at least:

	# truss -f /usr/obj/usr/src/arm.armv7/usr.bin/ldd/ldd -a 
"/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3"
	95910: 
mmap(0x0,135168,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 
1074327552 (0x4008f000)
	95910: mprotect(0x4007d000,4096,PROT_READ)	 = 0 (0x0)
	95910: issetugid()				 = 0 (0x0)
	95910: sigfastblock(0x1,0x4008df70)		 = 0 (0x0)
	95910: open("/etc/libmap.conf",O_RDONLY|O_CLOEXEC,01) = 3 (0x3)
	95910: fstat(3,{ mode=-rw-r--r-- ,inode=108965,size=47,blksize=4096 }) 
= 0 (0x0)
	95910: read(3,"# $FreeBSD$\nincludedir /usr/loc"...,47) = 47 (0x2f)
	95910: close(3)					 = 0 (0x0)
	95910: 
open("/usr/local/etc/libmap.d",O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC,0145) 
ERR#2 'No such file or directory'
	95910: open("/var/run/ld-elf.so.hints",O_RDONLY|O_CLOEXEC,0204411) = 3 
(0x3)
	95910: read(3,"Ehnt\^A\0\0\0\M^@\0\0\0|\0\0\0\0"...,128) = 128 (0x80)
	95910: fstat(3,{ mode=-r--r--r-- ,inode=270241,size=252,blksize=4096 }) 
= 0 (0x0)
	95910: pread(3,"/lib:/usr/lib:/usr/lib/compat:/u"...,124,0x80) = 124 
(0x7c)
	95910: close(3)					 = 0 (0x0)
	95910: 
open("/lib/libelf.so.2",O_RDONLY|O_CLOEXEC|O_VERIFY,010002250025) = 3 
(0x3)
	95910: fstat(3,{ mode=-r--r--r-- ,inode=109043,size=88428,blksize=88576 
}) = 0 (0x0)
	95910: mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 
1074028544 (0x40046000)
	95910: mmap(0x0,282624,PROT_NONE,MAP_GUARD,-1,0x0) = 1074462720 
(0x400b0000)
	95910: 
mmap(0x400b0000,12288,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) 
= 1074462720 (0x400b0000)
	95910: 
mmap(0x400c2000,77824,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x2000) 
= 1074536448 (0x400c2000)
	95910: 
mmap(0x400e4000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x14000) 
= 1074675712 (0x400e4000)
	95910: 
mmap(0x400f4000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x14000) 
= 1074741248 (0x400f4000)
	95910: munmap(0x40046000,4096)			 = 0 (0x0)
	95910: close(3)					 = 0 (0x0)
	95910: 
open("/lib/libgcc_s.so.1",O_RDONLY|O_CLOEXEC|O_VERIFY,010002250027) = 3 
(0x3)
	95910: fstat(3,{ mode=-r--r--r-- ,inode=109245,size=44108,blksize=44544 
}) = 0 (0x0)
	95910: mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 
1074028544 (0x40046000)
	95910: mmap(0x0,241664,PROT_NONE,MAP_GUARD,-1,0x0) = 1074745344 
(0x400f5000)
	95910: 
mmap(0x400f5000,12288,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) 
= 1074745344 (0x400f5000)
	95910: 
mmap(0x40107000,36864,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x2000) 
= 1074819072 (0x40107000)
	95910: 
mmap(0x4011f000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0xa000) 
= 1074917376 (0x4011f000)
	95910: 
mmap(0x4012f000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0xa000) 
= 1074982912 (0x4012f000)
	95910: munmap(0x40046000,4096)			 = 0 (0x0)
	95910: close(3)					 = 0 (0x0)
	95910: open("/lib/libc.so.7",O_RDONLY|O_CLOEXEC|O_VERIFY,010002250023) 
= 3 (0x3)
	95910: fstat(3,{ mode=-r--r--r-- 
,inode=109132,size=1708716,blksize=131072 }) = 0 (0x0)
	95910: mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 
1074028544 (0x40046000)
	95910: mmap(0x0,2056192,PROT_NONE,MAP_GUARD,-1,0x0) = 1074987008 
(0x40130000)
	95910: 
mmap(0x40130000,294912,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) 
= 1074987008 (0x40130000)
	95910: 
mmap(0x40187000,1388544,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x47000) 
= 1075343360 (0x40187000)
	95910: 
mmap(0x402e9000,20480,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x199000) 
= 1076793344 (0x402e9000)
	95910: 
mmap(0x402fd000,16384,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x19d000) 
= 1076875264 (0x402fd000)
	95910: 
mmap(0x40301000,151552,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,-1,0x0) 
= 1076891648 (0x40301000)
	95910: munmap(0x40046000,4096)			 = 0 (0x0)
	95910: close(3)					 = 0 (0x0)
	95910: mprotect(0x402e9000,16384,PROT_READ)	 = 0 (0x0)
	95910: sysarch(ARM_SET_TP,0x4009f010)		 = 0 (0x0)
	95910: __sysctl("hw.10",2,0x40303698,0xffffcc78,0x0,0) ERR#2 'No such 
file or directory'
	95910: readlink("/etc/malloc.conf",0xffffc847,1024) ERR#2 'No such file 
or directory'
	95910: issetugid()				 = 0 (0x0)
	95910: 
mmap(0x0,2097152,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(21),-1,0x0) 
= 1077936128 (0x40400000)
	95910: clock_gettime(4,{ 80683.109657689 })	 = 0 (0x0)
	95910: clock_gettime(4,{ 80683.109707855 })	 = 0 (0x0)
	95910: clock_gettime(4,{ 80683.109765151 })	 = 0 (0x0)
	95910: 
mmap(0x0,20480,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074028544 (0x40046000)
	95910: 
mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074049024 (0x4004b000)
	95910: 
openat(AT_FDCWD,"/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3",O_RDONLY|O_VERIFY,00) 
= 3 (0x3)
	95910: fstat(3,{ mode=-rwxr-xr-x ,inode=3021,size=1752272,blksize=4096 
}) = 0 (0x0)
	95910: mmap(0x0,1752272,PROT_READ,MAP_PRIVATE,3,0x0) = 1080033280 
(0x40600000)
	95910: 
mmap(0x0,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074053120 (0x4004c000)
	95910: 
mmap(0x0,28672,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074192384 (0x4006e000)
	95910: 
mmap(0x0,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074065408 (0x4004f000)
	95910: munmap(0x40600000,1752272)		 = 0 (0x0)
	95910: close(3)					 = 0 (0x0)
	95910: 
mmap(0x0,20480,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074221056 (0x40075000)
	95910: 
mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074077696 (0x40052000)
	95910: 
mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074081792 (0x40053000)
	95910: 
mmap(0x0,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074241536 (0x4007a000)
	95910: 
mmap(0x0,20480,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074257920 (0x4007e000)
	95910: 
mmap(0x0,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074278400 (0x40083000)
	95910: 
mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074085888 (0x40054000)
	95910: 
mmap(0x0,28672,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) 
= 1074290688 (0x40086000)
	95911: <new process>
	95910: fork()					 = 95911 (0x176a7)
	95911: execve("/libexec/ld-elf.so.1",0xffffda78,0x40054000) EJUSTRETURN
	95911: 
mmap(0x0,135168,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 
1074126848 (0x4005e000)
	95911: mprotect(0x4e000,4096,PROT_READ)		 = 0 (0x0)
	95911: issetugid()				 = 0 (0x0)
	95911: 
open("/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3",O_RDONLY|O_CLOEXEC|O_VERIFY,00) 
= 3 (0x3)
	95911: fstat(3,{ mode=-rwxr-xr-x ,inode=3021,size=1752272,blksize=4096 
}) = 0 (0x0)
	95911: geteuid()				 = 0 (0x0)
	95911: sigfastblock(0x1,0x5ef70)		 = 0 (0x0)
	95911: mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 
1074262016 (0x4007f000)
	95911: 
mmap(0x10000,1949696,PROT_NONE,MAP_FIXED|MAP_GUARD|MAP_EXCL,-1,0x0) 
ERR#12 'Cannot allocate memory'
	95911: munmap(0x4007f000,4096)			 = 0 (0x0)
	95911: close(3)					 = 0 (0x0)
	ld-elf.so.1: 95911: write(2,"ld-elf.so.1: ",13)		 = 13 (0xd)
	/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3: 
mmap of entire address space failed: Cannot allocate memory95911: 
write(2,"/wrkdirs/usr/ports/databases/sql"...,138) = 138 (0x8a)

	95911: write(2,"\n",1)				 = 1 (0x1)
	95911: exit(0x1)
	95911: process exit, rval = 1
	95910: wait4(-1,{ EXITED,val=1 },0x0,0x0)	 = 95911 (0x176a7)
	/wrkdirs/usr/ports/databases/sqlite3/work-default/stage/usr/local/bin/sqlite3: 
exit status 1
	95910: write(2,"/wrkdirs/usr/ports/databases/sql"...,93) = 93 (0x5d)
	95910: exit(0x1)
	95910: process exit, rval = 1

Best regards,
Kristof