From nobody Fri May 16 17:18:29 2025 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZzYh82s2Hz5w2qt for ; Fri, 16 May 2025 17:18:32 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZzYh80v1Bz3nDJ; Fri, 16 May 2025 17:18:32 +0000 (UTC) (envelope-from avg@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1747415912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=36UFuBE1BfpzY9zoIiWA3B/H3hb7U+EJlQ5yHO35UGU=; b=bYM+2b/xL9UZa1cKkSPQMzeaZri5y2WFmbr4c8rnSAIcdGUzMhN8LSB86Gj0m7bse8SrIv 4V13u1rL+tpLdozjgSUOb8c9Uyaa1QmD2NeC9TvPsLwTwPVHO1dpoUiYsoSN6/UhpoP4AW ROJr9tP+gY9i4gffJSgoYxRsVUJMLZkQWAlebAtntSfRCohrXLlqx6Att/Ms2QbwdGFocN ndwk6UvPsHtiChg6q5bRxXhYZFKrDtOTcsN1dWUrxrNIwQ8SSF5OXrbczWJyegOwGbT5lN kXlQnu39Hm3opRxfpzqvz0ex286EFjxeBQ6ivafSyTMFExG6hxbGYR65NUj9Tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1747415912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=36UFuBE1BfpzY9zoIiWA3B/H3hb7U+EJlQ5yHO35UGU=; b=Gp2Ul3byxN9o7J7wzA05niTHwWta5o4z1JQebHxPDvbH8Q7Jds4g9tTtc9QLdJU11Vnvs8 RKnhSv67y9rENzbhoEgk4F0mK8Gwhon48B+pmouep1CI6KlzPHMOhY0eUNWtifSKBIivOm 1AWdk6iFCaO+Qhbwn8RQ7fHt3YZ+EQpsUYhSNe2Xbj1avTXytnziQEOZeKjH/7oe1OGI/t dv4hJVALhxektaXwNW1aRH5j4Tfe0DL4EhP79N/zZUb1wt+niFWCmnmg4GlcDkBiuy4YZX 8B9xfqL/XNXwU2fiZFlDlEB+o7scLkPtjUBgLr0y8g7pGButnf0CbheBeMXd5w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1747415912; a=rsa-sha256; cv=none; b=OG1T8aEAqaemBEcPKQLUw25WpLIiLSzmIivlkmey3Tqa5WuxFzE5ymQr3ZTuFmoNfVd0xA dMCL5EhdXI/BmhXwvfk/6BtU3KsnCje3in5as9L3DR8Hj9Uw066KBmIOjO1TOClJJpocJY eKNbWbu13AyOdUB969H483rLHv1gGcsNDQv7SeZ1LvzUZzM0yI+k/bLHgJvyCfVFNu+oMR 4EDySrHlAF13iAXFZLtgC4hWgdzdRX9OicLYAvLbWi8r3AjJhpSzydYPCLuKXgxX0g5RBr U0n8jpGVqZjoumc4z8pVhYSlWitl+3buLJqoUwpiwLu30rMVzjKH3xcGxPpC/w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from [192.168.0.88] (unknown [93.188.39.137]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: avg/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4ZzYh73rh3z1003; Fri, 16 May 2025 17:18:31 +0000 (UTC) (envelope-from avg@freebsd.org) Message-ID: Date: Fri, 16 May 2025 20:18:29 +0300 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Andriy Gapon Subject: libdtrace symbol map [Was: RTLD_DEEPBIND question] To: Mark Johnston Cc: Konstantin Belousov , FreeBSD Current References: <900c8521-559a-47b5-acaa-ae941f6852c4@freebsd.org> <7c4e1682-d797-493c-8326-08d51dde3359@FreeBSD.org> <59db8ace-770f-4f73-976f-411f6de0885a@FreeBSD.org> <5b887290-a6aa-49ff-aa92-f335ed09b9a5@freebsd.org> Content-Language: en-US Autocrypt: addr=avg@freebsd.org; keydata= xsDNBGcKrHEBDADRvwQOK0b/yo4ys5cs6bOQMhEh4xtfbaZ/CU00cpPgUip3sOZCdrtMWlRC g25z97prxE9pKueZi+HXDhIPpa9xl14ghqF4oYScuJ1i18HyiOH2y5Q3Vv/TtFiSzicd3EAu QgS3jVidpgDSPDdj2Yz3UxYpZ+PuFl6nOnvCvqOFcjUlzKCyPaiN2b86l1Nscmhnc+zQ/faB erUOEFEDQbWMA5YfXi8HrbeR16hfRfGt7E0aMDlIj9FIPIq71UWMN9CimPgs4+rbNr1MAlLa z4GxSDhVYZEY5rqtCzr+PLXboRQWnaUwXl0/biw9enf17NHdYv1SNAFTX2eC4dZ3qBVI74dS PgNprm+PMfz+6Hhs/dAv+Nan5nVhg3EFIjYTiy0MnjMSq8uI0v0ykpAGAcJJ5xl6d23aLxgN 6f0z6pJRCO0hGPgU7UzvFD0MxJxmbzqdT1R51KDan1oD41b+tjl2LMBuCDCoB0U44Pu0zLdp xMfFTxCXtwIYKIUxwd28jwMAEQEAAc0eQW5kcml5IEdhcG9uIDxhdmdARnJlZUJTRC5vcmc+ wsENBBMBCAA3FiEEmXvSmjiQFHPVOpLnzDOt5NLj67sFAmcKrHEFCQeEzgACGwMECwkIBwUV CAkKCwUWAgMBAAAKCRDMM63k0uPru5tSDACFK15LLbq89RSQ6QMnjiIm1t/wYJyumb519MHu Dhzxx1lbr8oghf0RHtF6kYRLQPaW2VdToi74pRobd3CN4bhZKDLSL6WfTn17RfavDjL6Njwp KBo30CkOeYKWq1mDmo0xEoQj8cc7ybEZnus+YScZOpj8Ti4EFwhRt6SHer7YDb161IHKL8m4 MsCxpFSGEjbKj8Iul3Ri/fTOO8w14ivcuEEQIvJt4/+4YV5Az8G23wKzL/3aJ7SOT3oYGmR9 atBTmVO3DlODjM+rZLegd8SfLSPTcBTHspWE5duemIzZbEX3BP77r3Qx4Fo5Tkit3bG1XVar yPQato+sFGFEGifdE9USBQoAoOaaeZevwAWjDU0TIuCT0CUe0sKtQuNP4LRq0n9EEHOXBu9a CfdMhFUSkAZnuE7miSVwgPvoVNJ1stA37EXLN/sVsWik7wslTQ5vF81VpdGFiwoQPOe2XEKh ogcwGSnXbwv1gD4x+Gz/7Y+kFyr1NY+4/nSaeXVcS2fOwM0EZwqscgEMAMQTe6ypAmQe/TFO HqKD2hfFKdksTptKi6uEh8xIwct8G/0FBldDWXo9eu8CGr/ZrDg0/bAwJxbaLRQCMH19Gq2Y hLvZ1QK5GQJVzZKcqfxbF2LiDUTs6WkdOBIhGpdDy7p1xFrvqCGCtNFYHuGYm067EozibBSF BWAPstKu2FQuVHZNMOfs7p3OIz3Yfqu9woXDeg3/8G2qVQJINe+8EwXKlhgh4CyDbq7nAZoA kIu1SE9z9u3WI5mcNy/0dFmVUsFxBqRC3ewbvzie8tKyZ9yFOlaZPT0Y4nRBXQTI3mLZ8zQ8 mtrWK5OOmrJ02kdeO9RBXe+OMaUUWMf92ZIoBFb4HP6N+B+4N1y1OwULousfl7JRoYxA4MRL ls7E2sSoJvrEBTJB3Pc34xu8rsJ1A5V3NgN6djX8yEZYpTRkcmrBeWy/ofDqZPVqneAx0LRm eldDS9msXDW4KXODyPZ+9unvmHAcoH0xaBYaSH44CDZDQDg4LNcmbOvuu1TEXBJhjQARAQAB wsD8BBgBCAAmFiEEmXvSmjiQFHPVOpLnzDOt5NLj67sFAmcKrHMFCQeEzgACGwwACgkQzDOt 5NLj67sUCAv5AXqgWnYN9EblapMbZjkiqL8pZQ0GNqh+Pg9FwbyULxjtRTO6rD4D0IxizByb ef+neeUNyYlagt5nfKMysEr0SU/gHKCi8vyTF/63ukMrGUNGmJJxrndl5ZYKC6j6eX7twrZF L1Uvlmn6FnQ22red5kHO93fDjG4zaDIZvHfwj7kzjZ4tpC7Byinf88s14mdZeScc0PnU2hj4 UGYju/wg2FF4YxaZYhcmdTiRYY0Wx85XSMZv19pnn78sadEuRvfRd4JTmw++j1xGXeqQGWzz /CTG5/Ex9GAkQ02hZbmi236byDXoet4G8TEyOph9QFVkV9bNd0jQZaFZPGEj4PSPUYGAF7s5 xJaNGgctC3aZ7WjEv1FBoo44XCU4xcjJ1wZQUrHxRhx6TW0Jtcl0U9qfKFW30TSPo6RyiXuj X4ltWKAtjoXB8nUmEJckaz7IRu2b4pXDeazZuz5JBygUs10yJjDxh2vFQZo0KaBAPx9MZlPn gpPTjT15L8xGftEjQXF6 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 16/05/2025 17:11, Mark Johnston wrote: > On Fri, May 16, 2025 at 03:22:52PM +0300, Andriy Gapon wrote: >> On 27/04/2025 17:26, Mark Johnston wrote: >>> On Thu, Apr 24, 2025 at 08:56:44AM +0300, Andriy Gapon wrote: >>>> On 23/04/2025 21:56, Andriy Gapon wrote: >>>>> BTW, I've been wondering how illumos avoids the problem even though they >>>>> do not use any special dlopen flags. >>>>> It turns out that they link almost all system shared libraries with >>>>> -Bdirect option (which is Solaris/illumos specific). >>>>> It's somewhat similar to, but different from, -Bsymbolic. >>>>> https://docs.oracle.com/cd/E23824_01/html/819-0690/aehzq.html#scrolltoc >>>>> https://docs.oracle.com/cd/E36784_01/html/E36857/gejfe.html >>>> >>>> Oh, and it looks like there is an even better explanation for illumos. >>>> There is a version map file for libdtrace which explicitly lists API >>>> functions and makes everything else local. >>>> https://github.com/illumos/illumos-gate/blob/master/usr/src/lib/libdtrace/common/mapfile-vers >>>> >>>> I wonder why we didn't do the same when porting. >>>> Maybe we should do that now? >>> >>> I don't have any objection, but I believe adding a version map after the >>> fact doesn't accomplish much, assuming that we care deeply about ABI >>> stability in libdtrace.so (I'm not sure we do though). >> >> My primary goal here is not ABI stability, but hiding symbols that really >> should not be exported. See more at the end. >> >> At the same time I am not sure why it could be too late to start caring >> about ABI stability now. Assuming we actually would want to do that. > > I just mean that the version map helps only helps provide stability for > binaries linked to libdtrace.so after the version map is introduced. So, if we introduce it (and bump the library version as kib pointed out), then eventually it becomes useful (like it did for all other libraries that we versioned). >> And I don't want to single out libtrace here. >> It seems that the story is the same for all libraries that have been ported >> from illumos. >> E.g., libzfs_core was supposed to be a library that cares greatly about its >> API / ABI stability. >> >>>>> I think that on FreeBSD we should use symbol visibility attributes or a >>>>> symbol map to hide (make local) symbols that are not expected to be >>>>> interposed or have a high chance to be interposed by accident. >>>>> >>>>> IMO, yyparse should definitely get that treatment. >>>>> >>>>> I think that approach would be better than magic rtld tricks. >>>>> Especially because the tricks do not work with the current rtld. >>>>> I'd rather make a change to libdtrace.so than to rtld. >>>> >>>> This, while not as nice as the illumos solution, fixes my specific issue: >>>> diff --git a/cddl/lib/libdtrace/Makefile b/cddl/lib/libdtrace/Makefile >>>> index d086fffb07bc..58054d129b49 100644 >>>> --- a/cddl/lib/libdtrace/Makefile >>>> +++ b/cddl/lib/libdtrace/Makefile >>>> @@ -146,7 +146,8 @@ CFLAGS+= -fsanitize=address -fsanitize=undefined >>>> LDFLAGS+= -fsanitize=address -fsanitize=undefined >>>> .endif >>>> >>>> -LIBADD= ctf elf proc pthread rtld_db xo >>>> +VERSION_MAP= ${.CURDIR}/Symbol.map >>>> +LIBADD= ctf elf proc pthread rtld_db xo >>>> >>>> CLEANFILES= dt_errtags.c dt_names.c >>>> >>>> diff --git a/cddl/lib/libdtrace/Symbol.map b/cddl/lib/libdtrace/Symbol.map >>>> new file mode 100644 >>>> index 000000000000..89ee9de65209 >>>> --- /dev/null >>>> +++ b/cddl/lib/libdtrace/Symbol.map >>>> @@ -0,0 +1,4 @@ >>>> +{ >>>> + local: >>>> + yy*; >>>> +}; >>> >>> This just gives the lexer/parser symbols in libdtrace.so local >>> visibility? I think that's fine. >> Yes, that's the intention. >> >> I tested locally two versions of Symbol.map for libdtrace. >> The basic one quoted here and a more extended one based on illumos >> lib/libdtrace/common/mapfile-vers. >> The latter version does not define any symbol versions, its purpose is only >> to be a white-list of things to make public / global: >> https://people.freebsd.org/~avg/libdtrace-Symbol.map > > Do we really want to export _dtrace_debug? Hmm, I don't know, that came from illumos. However, I do not see any references to _dtrace_debug outside of libdtrace neither in FreeBSD nor in illumos. So, I guess that it can be removed. Maybe it was exposed for potential libdtrace consumers that might want to enable debug directly rather than via environment. >> Comparing to illumos I only had to add 3 dtrace_oformat* symbols, >> >> Both versions worked equally well in my tests, but maybe I missed more of >> FreeBSD extensions. >> >> Which one would be better to get into the tree? > > Having a full whitelist seems preferable to me. Did you test lockstat > as well? I believe it and dtrace(8) are the only users of libdtrace.so > in the base system. I didn't before, I've just done now, it works. I'll post a review request over the weekend (hopefully). LD_PRELOAD=/usr/obj/amd64.amd64/cddl/lib/libdtrace/libdtrace.so.2 lockstat -H -D 10 -x aggsize=100m -l smp_ipi_mtx -s 20 -P sleep 5 Spin lock hold: 106 events in 5.021 seconds (21 events/sec) ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Caller 11 53% 53% 0.00 99541 smp rendezvous __mtx_unlock_spin_flags+0xe3 nsec ------ Time Distribution ------ count Stack 16384 |@@ 1 smp_rendezvous_cpus+0x187 32768 |@@@@@@@@@@@@@@@@@@@ 7 dtrace_sync+0x39 65536 | 0 dtrace_state_deadman+0x13 131072 | 0 softclock_call_cc+0x1d9 262144 |@@ 1 softclock_thread+0xc6 524288 |@@@@@ 2 fork_exit+0x82 0xffffffff81030a3e ------------------------------------------------------------------------------- ... -- Andriy Gapon