From nobody Wed Jun 15 21:15:26 2022 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9CEBC834243 for ; Wed, 15 Jun 2022 21:15:31 +0000 (UTC) (envelope-from sysadmin.lists@mailfence.com) Received: from wilbur.contactoffice.com (wilbur.contactoffice.com [212.3.242.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LNdPZ3wSFz4bQn; Wed, 15 Jun 2022 21:15:30 +0000 (UTC) (envelope-from sysadmin.lists@mailfence.com) Received: from ichabod.co-bxl (ichabod.co-bxl [10.2.0.36]) by wilbur.contactoffice.com (Postfix) with ESMTP id 4D5F511B8; Wed, 15 Jun 2022 23:15:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1655327729; s=20210208-e7xh; d=mailfence.com; i=sysadmin.lists@mailfence.com; h=Date:From:Cc:Message-ID:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding; l=5993; bh=nBssYPRajN3WNugLJEwUdL7+EZJTNVRubEztJ9pmHsM=; b=s6IkGp13/b+dGPOJeKnNcYdXUFnNqnULYqRhEpTYEOC5+fjbMTQMRHrgtrT3LOHX ktwAgQ5xytlNjrn7Vfm+Uv+TYpRc0SxaVBj8xzSQTrATeSEP6tcdzIqtBGgU+9Ouly6 XpbVWNV+ggU6nG9cgS6khFf1ygNQl0wgLwbDrl0T5wlJcCedNFnmV0K2Xw3+E4pgFgI d5JTPsK7Isy6vjn3D20KIceNTKeBvKC57miqeE5YbfbNmmEXBvKm5IwtdDqSxUA+zRd z6t/FzHioxBfkrNptUOVNeiR90jrnCvfzxat7cEfObwGErC11SaWVbPn+29FwayGJ/j XRUNTcYdew== Date: Wed, 15 Jun 2022 23:15:26 +0200 (CEST) From: Sysadmin Lists To: freebsd-emulation@freebsd.org Cc: dchagin@freebsd.org, Martin Simmons Message-ID: <15732251.1658703.1655327726772@ichabod.co-bxl> In-Reply-To: <202206151944.25FJinlg024188@higson.cam.lispworks.com> References: <1236409007.1125724.1655169963971@ichabod.co-bxl> <2033861928.1354940.1655236558393@ichabod.co-bxl> <202206151704.25FH4m4d019069@higson.cam.lispworks.com> <606426103.1627955.1655317573324@ichabod.co-bxl> <202206151944.25FJinlg024188@higson.cam.lispworks.com> Subject: Re: libc6 Compatibility List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Mailer: ContactOffice Mail X-ContactOffice-Account: com:312482426 X-Rspamd-Queue-Id: 4LNdPZ3wSFz4bQn X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=fail ("body hash did not verify") header.d=mailfence.com header.s=20210208-e7xh header.b=s6IkGp13; dmarc=pass (policy=quarantine) header.from=mailfence.com; spf=pass (mx1.freebsd.org: domain of sysadmin.lists@mailfence.com designates 212.3.242.68 as permitted sender) smtp.mailfrom=sysadmin.lists@mailfence.com X-Spamd-Result: default: False [-3.87 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:212.3.242.64/26:c]; R_DKIM_REJECT(0.00)[mailfence.com:s=20210208-e7xh]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[mailfence.com:-]; DMARC_POLICY_ALLOW(0.00)[mailfence.com,quarantine]; NEURAL_HAM_SHORT(-0.98)[-0.981]; DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50)[]; MLMMJ_DEST(0.00)[freebsd-emulation]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:10753, ipnet:212.3.242.64/26, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_LOW(-0.10)[212.3.242.68:from] X-ThisMailContainsUnwantedMimeParts: N > ---------------------------------------- > From: Martin Simmons > Sent: Wed Jun 15 21:44:49 CEST 2022 > To: Sysadmin Lists > Cc: , > Subject: Re: libc6 Compatibility > > > >>>>> On Wed, 15 Jun 2022 20:26:13 +0200 (CEST), Sysadmin Lists said: > > > > > ---------------------------------------- > > > From: Martin Simmons > > > Sent: Wed Jun 15 19:04:48 CEST 2022 > > > To: Sysadmin Lists > > > Cc: , > > > Subject: Re: libc6 Compatibility > > > > > > > > > >>>>> On Tue, 14 Jun 2022 21:55:58 +0200 (CEST), Sysadmin Lists said: > > > > > > > > > ---------------------------------------- > > > > > From: Dmitry Chagin > > > > > Sent: Tue Jun 14 09:24:37 CEST 2022 > > > > > To: Sysadmin Lists > > > > > Cc: Freebsd Emulation > > > > > Subject: Re: libc6 Compatibility > > > > > > > > > > > > > > > On Tue, Jun 14, 2022 at 03:26:03AM +0200, Sysadmin Lists wrote: > > > > > > Does Linux compat not work with Ubuntu's newest libc6 update? > > > > > > I used to run Brave browser from it, but now I get this error message: > > > > > > > > > > > > $ /compat/ubuntu/opt/brave.com/brave/brave > > > > > > ELF interpreter /lib64/ld-linux-x86-64.so.2 not found, error 2 > > > > > > Abort trap > > > > > > > > > > > > The shared object is loaded in memory: > > > > > > $ ldd /opt/brave.com/brave/brave | grep ld- > > > > > > /lib64/ld-linux-x86-64.so.2 (0x0000000001021000) > > > > > > > > > > > > And it exists on the filesystem: > > > > > > $ find /compat/ubuntu/lib**/ -name ld-\* -exec ls -lh '{}' + > > > > > > -rwxr-xr-x 1 root wheel 187K Dec 16 2020 /compat/ubuntu/lib/x86_64-linux-gnu/ld-2.31.so > > > > > > lrwxr-xr-x 1 root wheel 10B Dec 16 2020 /compat/ubuntu/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 -> ld-2.31.so > > > > > > lrwxr-xr-x 1 root root 34B Mar 23 2021 /compat/ubuntu/lib64/ld-linux-x86-64.so.2 -> ../lib/x86_64-linux-gnu/ld-2.31.so > > > > > > > > > > > > This is the update that broke it: > > > > > > $ apt-get install libc6 > > > > > > Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libc6 amd64 2.31-0ubuntu9.9 [2722 kB] > > > > > > > > > > > > > > > > try ktrace it, aka ktrace -di /compat/ubuntu/opt/brave.com/brave/brave > > > > > then kdump -HAR -m 128 > brave.log > > > > > > > > > > > > > > > > > > Thanks, Dmitry. That helped me find the problem. Here's the full log: > > > > > > > > $ cat brave.log > > > > 8184 101222 ktrace 0.000000 RET F64 ktrace 0 > > > > 8184 101222 ktrace 0.000014 CALL F64 execve(0x7fffffffeda3,0x7fffffffeac0,0x7fffffffead0) > > > > 8184 101222 ktrace 0.000003 NAMI F64 "/compat/ubuntu/opt/brave.com/brave/brave" > > > > 8184 101222 ktrace 0.055726 NAMI F64 "/compat/ubuntu/lib64/ld-linux-x86-64.so.2" > > > > 8184 101222 ktrace 0.000032 NAMI F64 "/lib64/ld-linux-x86-64.so.2" > > > > > > > > And here's an updated `ls' printout after upgrading libc6: > > > > $ find /compat/ubuntu/lib**/ -name ld-\* -exec ls -lh '{}' + > > > > -rwxr-xr-x 1 root wheel 187K Apr 6 18:24 /compat/ubuntu/lib/x86_64-linux-gnu/ld-2.31.so > > > > lrwxr-xr-x 1 root wheel 10B Apr 6 18:24 /compat/ubuntu/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 -> ld-2.31.so > > > > lrwxr-xr-x 1 root wheel 32B Apr 6 18:24 /compat/ubuntu/lib64/ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.31.so > > > > > > > > The new .so is missing 'previous directory' dots in the symlink path. > > > > > > It looks like the link with the dots was made by hand in 2021, presumably to > > > fix this. > > > > That might be the case. I remember having to make changes to the shared object > > before. The question then becomes, why only this update to libc6 and not > > previous updates? > > Based on the file dates, I suspect there were no previous updates to libc6 > installed since Dec 16 2020. You must be right, but I could have sworn I've updated libc6 at least a couple times. Something caused me to recreate the symlink in Mar 2021. > > > > I thought Linux compat was designed to search compat.linux.emul_path > > > > for shared libs first, but apparently it works differently than that. > > > > > > What does linux compat do in general with absolute symlinks? Does it check > > > the target with compat.linux.emul_path prepended? > > > > That's what the documentation says: > > "Linux mode dynamically reroots lookups. This is, in effect, equivalent to > > union to file system mounts. First, an attempt is made to look up the file in > > /compat/linux/original-path. If that fails, the lookup is done in > > /original-path. This makes sure that binaries that require other binaries can > > run." > > https://docs.freebsd.org/en/books/handbook/linuxemu/#linuxemu-advanced > > > > Looking at the installer script, it's clear this is a known issue: > > $ sed -n '/fix_ld_path(/,/^$/p' /usr/local/share/linux-browser-installer/linux-browser-installer > > fix_ld_path() > > { > > (cd ${chroot_path}/lib64 && \ > > (unlink ./ld-linux-x86-64.so.2; \ > > ln -s ../lib/x86_64-linux-gnu/ld-${ld_version}.so \ > > ld-linux-x86-64.so.2)) > > } > > > > So, either: > > 1. The documentation is wrong. > > 2. It's a bug. > > Looking at how it is implemented, the rerooting applies to the argument to the > Linux system call before it calls the code that implements the FreeBSD system > call to do the actual work. I think the problem here is that symlinks are > resolved by FreeBSD system call code (the NAMI operation), but that doesn't do > any rerooting. > > I don't know if that is a bug or a necessary limitation. Interesting. Thanks for the added details and insight. > __Martin -- Sent with https://mailfence.com Secure and private email