From nobody Sun Feb 12 19:44:24 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PFHwr03lYz3r9XP for ; Sun, 12 Feb 2023 19:44:28 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com [IPv6:2607:f8b0:4864:20::832]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PFHwq1Mcxz3yvC for ; Sun, 12 Feb 2023 19:44:27 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=AL0GfG4t; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::832 as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org; dmarc=none Received: by mail-qt1-x832.google.com with SMTP id 5so11799166qtp.9 for ; Sun, 12 Feb 2023 11:44:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xLnT5ApllZ2ekvp8EoSe+7e07JM1n9NyH/7qQGXok+o=; b=AL0GfG4tTbGh6dCVF/jHOGJDZUva8gCsqZmVrNpCcR+dfmjH70dky0+MMdV/1eP6M5 6kqQGifHb2SS0hZgsRc2n1Zbwe0G4oUUvZ37T7HVTCwYKm4zlCySVfxy2WNzhJm4pV7T lJQF2dvvCv2o7w1mqrjpBw8EfS+wMH6JkZO/EB8Gk5XkR8foC7Q98UfapcalzU+/FxrM Ew8Q6+2rES02/QsnTlV243A6Qvc0Qvf1MZmk4pcWtfySibtEe5BAqf8GLf2cLtJ7RPkF Tc0CyTFXZfUorLz/uEYuLim7vB7+PMQZZau0L6k0fPFKyq+Jsc6LvThPRioQL+w4jn1j DLgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xLnT5ApllZ2ekvp8EoSe+7e07JM1n9NyH/7qQGXok+o=; b=GYFhDUI9YBYgnFfop8MFYAtbQsVgCdg12FkW8Gb6YEVTbecyup6b+OOn6rnsMzpBYs 3vbnW2ufAWsySPz0Uligeo28Nmu3u8nkW/B+ybSi1yRFUBLrTLQkKi1q3WI+SyNYD0I+ cs4qfLAvAgx+W34cFIPrflxUefiXYyy0HZ7KdG5jfnu1BPnUkcR0qhh4QKRfLYmHYSv5 73pByydjw2rLd0slBfEldM+j/fYGgx19W/yj11pW4OsXTGoFBdj/d5E0yEezBw2sRv2w RztgFxOgX2h3DOhPlKHTJwGwJBUkVGia2asfBbbxKw/x9gwiU3ddAe/otV7zAsbCasLK 7Vkw== X-Gm-Message-State: AO0yUKU5y41erLC8Tpy+wsnSkPu3tZx/ey2TqaW+Tm7szIkBzIovFElY r2p4SQ/ksMMWqEbtGLBsj/gJ6+8DX4MXMfKJ X-Google-Smtp-Source: AK7set9Zk1S8a/H04aSRkhX1jdv+IQpFVxWBWYKdx3oCoyv5mL7CuaYFepCk/T+sbR5g9r5kW03yKw== X-Received: by 2002:ac8:5554:0:b0:3b9:bc8c:c216 with SMTP id o20-20020ac85554000000b003b9bc8cc216mr19627567qtr.33.1676231066305; Sun, 12 Feb 2023 11:44:26 -0800 (PST) Received: from mutt-hbsd (pool-100-16-219-215.bltmmd.fios.verizon.net. [100.16.219.215]) by smtp.gmail.com with ESMTPSA id r21-20020ac85e95000000b003b835e7e283sm7886398qtx.44.2023.02.12.11.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Feb 2023 11:44:25 -0800 (PST) Date: Sun, 12 Feb 2023 14:44:24 -0500 From: Shawn Webb To: David Chisnall Cc: freebsd-hackers Subject: Re: CFT: snmalloc as libc malloc Message-ID: <20230212194424.ka7omg6ya3ain3ml@mutt-hbsd> X-Operating-System: FreeBSD mutt-hbsd 14.0-CURRENT-HBSD FreeBSD 14.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: <2f3dcda0-5135-290a-2dff-683b2e9fe271@FreeBSD.org> <20230210162350.gg5g7dihp3zef3ov@mutt-hbsd> <193D1E1C-73EB-42BB-8A6A-87B0E4AD9D7C@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="7emnhufbnpa7q373" Content-Disposition: inline In-Reply-To: <193D1E1C-73EB-42BB-8A6A-87B0E4AD9D7C@FreeBSD.org> X-Spamd-Result: default: False [-5.10 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::832:from]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[hardenedbsd.org]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4PFHwq1Mcxz3yvC X-Spamd-Bar: ----- X-ThisMailContainsUnwantedMimeParts: N --7emnhufbnpa7q373 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 11, 2023 at 05:10:02PM +0000, David Chisnall wrote: > On 10 Feb 2023, at 16:23, Shawn Webb wrote: > >=20 > > So I took a little bit of a different approach, which should provide > > the same end result as your submodule approach. Note that I'm doing > > this in HardenedBSD 14-CURRENT (the hardened/current/master branch). > >=20 > > 1. git cherry-pick -xs a5c83c69817d03943b8be982dd815c7e263d1a83 > > 2. git rm -f .gitmodules contrib/snmalloc > > 3. git commit > > 4. git subtree add -P contrib/snmalloc \ > > git@github.com:microsoft/snmalloc.git main > >=20 > > I believe this should leave me with a tree that populates > > contrib/snmalloc and pulls in your non-contrib/ changes, leading me to > > end up in the same end state as your submodule approach. > >=20 > > I am seeing some build errors. I've uploaded a WITHOUT_CLEAN=3Dyes log > > to: > >=20 > > https://hardenedbsd.org/~shawn/2023-02-10_snmalloc-01.log.txt > >=20 > > Note that this is with llvm 15.0.7 that just landed in FreeBSD main. >=20 > The error is a bit confusing, because nullptr_t has been in libc++ since = C++11. Does HardenedBSD change anything in include orders? Can you add -v= to the end of the compile command: >=20 >=20 > c++ -target x86_64-unknown-freebsd14.0 --sysroot=3D/usr/obj/data/src/har= denedbsd/amd64.amd64/tmp -B/usr/obj/data/src/hardenedbsd/amd64.amd64/tmp/us= r/bin -fomit-frame-pointer -O2 -pipe -fno-common -DHARDENEDBSD -DNO__SCCSID= -DNO__RCSID -I/data/src/hardenedbsd/lib/libc/include -I/data/src/hardenedb= sd/include -I/data/src/hardenedbsd/lib/libc/amd64 -DNLS -ftls-model=3Diniti= al-exec -D__DBINTERFACE_PRIVATE -I/data/src/hardenedbsd/contrib/gdtoa -I/da= ta/src/hardenedbsd/contrib/libc-vis -DINET6 -I/usr/obj/data/src/hardenedbsd= /amd64.amd64/lib/libc -I/data/src/hardenedbsd/lib/libc/resolv -D_ACL_PRIVAT= E -DPOSIX_MISTAKE -I/data/src/hardenedbsd/lib/libmd -I/data/src/hardenedbsd= /lib/libc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/data/src/hardenedb= sd/lib/libc/rpc -DWANT_HYPERV -DYP -DNS_CACHING -DSYMBOL_VERSIONING -g -gz= =3Dzlib -mretpoline -fPIC -flto -MD -MF.depend.malloc.o -MTmalloc.o -Wno-fo= rmat-zero-length -fstack-protector-strong -Wsystem-headers -Werror -Wall -W= no-format-y2k -Wno-uninitialized -Wdate-time -Wno-empty-body -Wno-string-pl= us-int -Wno-unused-const-variable -Wno-error=3Dunused-but-set-variable -Wno= -error=3Darray-parameter -Wno-error=3Ddeprecated-non-prototype -Wno-error= =3Dunused-but-set-parameter -Wno-tautological-compare -Wno-unused-value -Wn= o-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-unuse= d-local-typedef -Wno-address-of-packed-member -Wno-switch -Wno-switch-enum = -Wno-knr-promoted-parameter -Qunused-arguments -I/data/src/hardenedbsd/lib/= libutil -I/data/src/hardenedbsd/lib/msun/amd64 -I/data/src/hardenedbsd/lib/= msun/x86 --include-directory-after /data/src/hardenedbsd/lib/msun/src -DHAR= DENEDBSD -I/data/src/hardenedbsd/contrib/snmalloc/src/snmalloc -std=3Dc++2= 0 -mcx16 -fno-exceptions -fno-rtti -DSNMALLOC_USE_THREAD_CLEANUP -DSNMALLOC= _BOOTSTRAP_ALLOCATOR -DSNMALLOC_JEMALLOC3_EXPERIMENTAL -DSNMALLOC_JEMALLOC_= NONSTANDARD -DSNMALLOC_PLATFORM_HAS_GETENTROPY -DSNMALLOC_STATIC_LIBRARY_PR= EFIX=3D__ -ftls-model=3Dinitial-exec -DSNMALLOC_CHECK_CLIENT -DSNMALLOC_FAI= L_FAST=3Dfalse -DNDEBUG -g -gz=3Dzlib -mretpoline -flto -Wno-c++11-exten= sions -c /data/src/hardenedbsd/lib/libc/stdlib/snmalloc/malloc.cc -o mall= oc.o >=20 >=20 >=20 > That should dump the include paths. It=E2=80=99s possible that the libc+= + headers are being included in the wrong order with respect to the C paths? Hey David, HardenedBSD doesn't have any changes to userland that would cause changes in include header paths (and the priorities of them.) I made sure to `pkg delete -g gcc\* binutils\*` before running another buildworld, ending up with the same error. I tried running your command above. Here's the result: http://ix.io/4nS9 I think Mark might be on the right track with regards to llvm 15. I'll try importing the snmalloc patch (via the same subtree method) to a pre-llvm15 import branch and report back. It might be a few days before I get you a result. BTW, I appreciate the work. One long-term goal I've had for HardenedBSD is to be able to switch between multiple heap implementations at buildworld-time. I think it would be nifty for users to have a choice of jemalloc, snmalloc, and GrapheneOS' hardened_malloc. Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --7emnhufbnpa7q373 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmPpQZEACgkQ/y5nonf4 4frmqQ//eWViZtBrwkq3zrzEBbBEyxpWpQvzyG68scSvtdJ6/+bVzGbk/xR3mzso DDiWwaL2ehEBdWwj1WDS4bPKF33Tm8jdOje8gf0qG+ajgcnhSpZFTqNbTmOxWdNO YIKJkCuhxP4AOMD8mhEgdzZseOlYcO7txlFefI8e6+B9nc4siM7OmO76CgGrzN1Q MP9YPDAhvEI5txsRHh/BXPxDU5hk/stHsdnwdIrZn5hq65qXU49+omMxx/T62JQ9 fpapv2rZOc0PmkyrMuAQlKo6EnSzANG2Cudd5Bx4MI8oZbpCwZKXGIuOmYwgN1U5 PjOYJ0J4IPyU7u3nwerKAB9RpG+HNFgt4KJlXvc/YRi7732GcCKamcpXUkYmZWXC 3l/1tQH8NtX/quXwH5tOUI2PWjxRHJoHfRv1Dtw3hzzB5FEVNxa8A4D4TtH7EsJU oW81yfuFW59wLmLlnXV2u77/uyaoafEW/gxUjWzc8+9c4o1ZQ7Fogj0nf7cT8g0P Q13N92pEEtwF863uJ4X1VQtXlvDBYAa1rMfYGMNMrqrhkO0QwhY5J2krmM7+tSzP 4mpT1fPwHYg64gRJpoOKDQrSs9PvybmfxXht1wXHg5j9exJS4CgeCBuBj5D5+9rp 39oIeJvAKwk/f0zdi6EvhNBTjwV3hYAqGalYvgix4csASIDiAyA= =blgy -----END PGP SIGNATURE----- --7emnhufbnpa7q373--