Re: AMD64 14.0-CURRENT memory layout changes
- Reply: Paul Floyd : "Re: AMD64 14.0-CURRENT memory layout changes"
- In reply to: Paul Floyd : "AMD64 14.0-CURRENT memory layout changes"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Sun, 16 Oct 2022 15:23:52 UTC
On Sun, Oct 16, 2022 at 05:08:30PM +0200, Paul Floyd wrote: > Hi > > I just noticed that the memory layout has changed for elf binaries > running on amd64 (my last attempt to setup an i386 VM failed so I can't > confirm if that also changed, and I'm not yet concerned by other platforms). > > Here's a procstat -v for ksh93 on 13.1 on the host machine > > > paulf> procstat -v 1456 > > PID START END PRT RES PRES REF SHD FLAG TP PATH > > 1456 0x200000 0x273000 r-- 70 343 30 16 CN--- vn /usr/local/bin/ksh93 > > 1456 0x273000 0x3d1000 r-x 257 343 30 16 CN--- vn /usr/local/bin/ksh93 > > 1456 0x3d1000 0x3dd000 r-- 11 0 1 0 CN--- vn /usr/local/bin/ksh93 > > 1456 0x3dd000 0x3de000 rw- 1 0 1 0 CN--- vn /usr/local/bin/ksh93 > > 1456 0x3de000 0x3e4000 rw- 6 0 1 0 C---- vn /usr/local/bin/ksh93 > > 1456 0x3e4000 0x3ec000 rw- 5 5 1 0 C---- df > > 1456 0x8003de000 0x8003e4000 r-- 6 28 364 120 CN--- vn /libexec/ld-elf.so.1 > > 1456 0x8003e4000 0x8003fb000 r-x 23 28 364 120 CN--- vn /libexec/ld-elf.so.1 > > 1456 0x8003fb000 0x8003fc000 r-- 1 0 1 0 CN--- vn /libexec/ld-elf.so.1 > > 1456 0x8003fc000 0x800411000 rw- 19 19 1 0 CN--- df > > 1456 0x800420000 0x800432000 r-- 13 33 352 176 CN--- vn /lib/libm.so.5 > > 1456 0x800432000 0x800459000 r-x 20 33 352 176 CN--- vn /lib/libm.so.5 > > 1456 0x800459000 0x80045a000 rw- 1 0 1 0 CN--- vn /lib/libm.so.5 > > 1456 0x80045a000 0x80045b000 rw- 1 0 1 0 CN--- vn /lib/libm.so.5 > > 1456 0x80045b000 0x8004df000 r-- 132 380 528 284 CN--- vn /lib/libc.so.7 > > 1456 0x8004df000 0x80062b000 r-x 234 380 528 284 CN--- vn /lib/libc.so.7 > > 1456 0x80062b000 0x800633000 r-- 8 0 1 0 CN--- vn /lib/libc.so.7 > > 1456 0x800633000 0x800634000 rw- 1 0 1 0 CN--- vn /lib/libc.so.7 > > 1456 0x800634000 0x80063b000 rw- 7 0 1 0 C---- vn /lib/libc.so.7 > > 1456 0x80063b000 0x8008f5000 rw- 84 84 1 0 C---- df > > 1456 0x800a00000 0x801200000 rw- 13 13 1 0 CN--- df > > 1456 0x7fffdffff000 0x7ffffffdf000 --- 0 0 0 0 ----- gd > > 1456 0x7ffffffdf000 0x7ffffffff000 rw- 11 11 1 0 C--D- df > > 1456 0x7ffffffff000 0x800000000000 r-x 1 1 125 0 ----- ph > > Here the stack starts at 0x7ffffffdf000 > > And the same on 14.0 running on a 4Gbyte VirtualBox VM > > > paulf@freebsd:~/valgrind $ procstat -v 62770 > > PID START END PRT RES PRES REF SHD FLAG TP PATH > > 62770 0x200000 0x273000 r-- 115 488 4 2 CN--- vn /usr/local/bin/ksh93 > > 62770 0x273000 0x3c7000 r-x 340 488 4 2 CN--- vn /usr/local/bin/ksh93 > > 62770 0x3c7000 0x3d4000 r-- 13 0 2 0 C---- vn /usr/local/bin/ksh93 > > 62770 0x3d4000 0x3d5000 rw- 1 0 2 0 C---- vn /usr/local/bin/ksh93 > > 62770 0x3d5000 0x3da000 rw- 5 0 1 0 C---- vn /usr/local/bin/ksh93 > > 62770 0x3da000 0x3e2000 rw- 5 5 1 0 ----- sw > > 62770 0x80075d000 0x82073d000 --- 0 0 0 0 ----- gd > > 62770 0x82073d000 0x82075d000 rw- 14 14 1 0 ---D- sw > > 62770 0x8209c8000 0x8209c9000 r-x 1 1 28 0 ----- ph > > 62770 0x8217b0000 0x8217c2000 rw- 16 16 1 0 ----- sw > > 62770 0x822186000 0x822210000 r-- 138 496 104 54 CN--- vn /lib/libc.so.7 > > 62770 0x822210000 0x82235e000 r-x 334 496 104 54 CN--- vn /lib/libc.so.7 > > 62770 0x82235e000 0x822367000 r-- 9 0 2 0 C---- vn /lib/libc.so.7 > > 62770 0x822367000 0x822368000 rw- 1 0 2 0 C---- vn /lib/libc.so.7 > > 62770 0x822368000 0x82236f000 rw- 7 0 1 0 C---- vn /lib/libc.so.7 > > 62770 0x82236f000 0x82259e000 rw- 20 20 1 0 ----- sw > > 62770 0x823434000 0x823447000 r-- 19 59 4 2 CN--- vn /lib/libm.so.5 > > 62770 0x823447000 0x82346f000 r-x 40 59 4 2 CN--- vn /lib/libm.so.5 > > 62770 0x82346f000 0x823470000 rw- 1 0 1 0 C---- vn /lib/libm.so.5 > > 62770 0x823470000 0x823471000 rw- 1 0 1 0 C---- vn /lib/libm.so.5 > > 62770 0x823e0e000 0x823e3e000 rw- 16 16 1 0 ----- sw > > 62770 0x824600000 0x824800000 rw- 11 11 1 0 ----- sw > > 62770 0x8251a1000 0x8253a1000 rw- 1 1 1 0 ----- sw > > 62770 0x825e00000 0x826200000 rw- 3 3 1 0 ----- sw > > 62770 0x826a49000 0x826a61000 rw- 8 8 1 0 ----- sw > > 62770 0x826e5c000 0x826e74000 rw- 14 14 1 0 ----- sw > > 62770 0x827d6e000 0x827d86000 rw- 9 9 1 0 ----- sw > > 62770 0x8288ba000 0x8288d2000 rw- 5 5 1 0 ----- sw > > 62770 0x8296db000 0x8296f3000 rw- 3 3 1 0 ----- sw > > 62770 0xeeeecc15000 0xeeeecc1b000 r-- 6 29 71 21 CN--- vn /libexec/ld-elf.so.1 > > 62770 0xeeeecc1b000 0xeeeecc32000 r-x 23 29 71 21 CN--- vn /libexec/ld-elf.so.1 > > 62770 0xeeeecc32000 0xeeeecc33000 r-- 1 0 1 0 C---- vn /libexec/ld-elf.so.1 > > 62770 0xeeeecc33000 0xeeeecc35000 rw- 2 2 1 0 ----- sw > > 62770 0x7ffffffff000 0x800000000000 --- 0 0 0 0 ----- gd > > > ldrt is now mapped up at 0xeeeecc15000 and the user stack looks like it > starts at 0x82073d000. > > This is causing me problems with Valgrind, which creates the guest stack > at 0x7ffffffdf000. > > I haven't yet done any debugging of the problem but this causes > > Fatal error 'Cannot allocate red zone for initial thread' at line 395 in > file /usr/src/lib/libthr/thread/thr_init.c (errno = 22) > > for elf binaries linked with libthr.so > > Can anyone point me to more information on this change? Phabricator for > instance. It is related to some changes to randomize the base of the process stack, as we do for other components of the address space. See https://cgit.freebsd.org/src/commit/?id=1811c1e957ee1250b08b3246fc0db37ddf64b736 for example. In 14.0 randomization is enabled by default. > Are there any syscalls that control where rtld gets loaded and/or where > the stack base is located? No, their locations are randomized. There are sysctl knobs for controlling whether randomization is enabled or not. > Also is there a sysctl to disable this changed mapping, as a temporary > workaround? Setting kern.elf(64|32).aslr.stack to 0 should restore the old behaviour. It should also be possible to disable this on a per-process basis with proccontrol(1), but that doesn't appear to work, i.e., there is a bug. However, all randomization can be disabled this way, try "procstat -m aslr -s disable ksh93".