From nobody Sun Feb 12 21:09:04 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PFKpV5YBvz3rLFF for ; Sun, 12 Feb 2023 21:09:06 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PFKpV3YqZz4GBJ for ; Sun, 12 Feb 2023 21:09:06 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x82f.google.com with SMTP id f10so11977303qtv.1 for ; Sun, 12 Feb 2023 13:09:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=H/VqqQqqtAbnEpGdzeUXSLEqpNj1jwY7elsY7gSSCiA=; b=gBO+JDd+c13AvVfR3dxYxA+r+vyPGQEl3d80vx8Fc8qEKnOZojHXZIxKMog/SI1f5w aIkYcVGTcW84tzy4PWG9zdl3w4u+tJScd5uHWFMiV21Kg26+dHc6iPyPAQ3uG4ulslql IuPgTE8aupkZyeGiX/jI6cHN1PnxV64w7z6HOb5xvERFO1H6ZMcKKZ9qGpXBOUyohC2F gZJqD7LEMN50KLxxHEq3Ffw5jmY3Yb6g2Dpig+0GsE5Ij8KNUWPKCya38zVqZiRvNGrz xxeqC5y9WMeU5xDms5j14LP4jE0OWKsh9WTtR8e9cFMUSD8fg3n1btiL5R+oJ00j0K/a sPkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=H/VqqQqqtAbnEpGdzeUXSLEqpNj1jwY7elsY7gSSCiA=; b=OFLBsIgIleJn6/t9fhCVTrBuYvc/YwkYls/tyTGa6QYYRZdQI3dUZAkfDm545mWI35 jZ+BHX9KGpbzx7f8JV1c3ARSAWk0LrdrNcfxbnhC0QgWISLkitKFhMu++VxexbhQeAEK jsND5ukaDFOl1Ro1bfwvbYon6jXHQBpBvDOHbRpvnW5FsVBR9c4jRNelOoo3aWGtd1BU 90xjOge//25fTsUFfv41239Egao7PQMzEHQjq36pkPTxTj4Ga5GTg7AH3nOh+DjcNCQ6 71cLYe5yAWgsDeyo677b3kO4pXhk4DAKnDpwP7KDDSH0kRdfQDFeuMjjs9kof7WJYacg DVAQ== X-Gm-Message-State: AO0yUKU+h9YfK4HthS23fpAmIHLLIGBwJc0S2/x8nk6dyoLs34tfZPiN SpzSriXSuSl2MadPD+miMgOhlg== X-Google-Smtp-Source: AK7set+HNKPdsSX4NfZwDiOqwv8JskxQuUh5pGCsLo7i2omCGvQIZfj7xktt1ctBwfV7gA5SBUrfoA== X-Received: by 2002:ac8:5fd3:0:b0:3b8:6930:ee6 with SMTP id k19-20020ac85fd3000000b003b869300ee6mr40052755qta.21.1676236145748; Sun, 12 Feb 2023 13:09:05 -0800 (PST) Received: from mutt-hbsd (pool-100-16-219-215.bltmmd.fios.verizon.net. [100.16.219.215]) by smtp.gmail.com with ESMTPSA id m190-20020a3758c7000000b0071d2cd07560sm8335906qkb.124.2023.02.12.13.09.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Feb 2023 13:09:05 -0800 (PST) Date: Sun, 12 Feb 2023 16:09:04 -0500 From: Shawn Webb To: Mark Millard Cc: FreeBSD Hackers , David Chisnall Subject: Re: CFT: snmalloc as libc malloc Message-ID: <20230212210904.nzxfwtzsjf2tu6ky@mutt-hbsd> X-Operating-System: FreeBSD mutt-hbsd 14.0-CURRENT-HBSD FreeBSD 14.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="utl62kziietstoa5" Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4PFKpV3YqZz4GBJ X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --utl62kziietstoa5 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Feb 12, 2023 at 12:01:22PM -0800, Mark Millard wrote: > Shawn Webb wrote on > Date: Sun, 12 Feb 2023 19:44:24 UTC : >=20 > > On Sat, Feb 11, 2023 at 05:10:02PM +0000, David Chisnall wrote: > > > On 10 Feb 2023, at 16:23, Shawn Webb wro= te: > > > >=20 > > > > So I took a little bit of a different approach, which should provide > > > > the same end result as your submodule approach. Note that I'm doing > > > > this in HardenedBSD 14-CURRENT (the hardened/current/master branch). > > > >=20 > > > > 1. git cherry-pick -xs a5c83c69817d03943b8be982dd815c7e263d1a83 > > > > 2. git rm -f .gitmodules contrib/snmalloc > > > > 3. git commit > > > > 4. git subtree add -P contrib/snmalloc \ > > > > git@github.com:microsoft/snmalloc.git main > > > >=20 > > > > I believe this should leave me with a tree that populates > > > > contrib/snmalloc and pulls in your non-contrib/ changes, leading me= to > > > > end up in the same end state as your submodule approach. > > > >=20 > > > > I am seeing some build errors. I've uploaded a WITHOUT_CLEAN=3Dyes = log > > > > to: > > > >=20 > > > > https://hardenedbsd.org/~shawn/2023-02-10_snmalloc-01.log.txt > > > >=20 > > > > Note that this is with llvm 15.0.7 that just landed in FreeBSD main. > > >=20 > > > The error is a bit confusing, because nullptr_t has been in libc++ si= nce C++11. Does HardenedBSD change anything in include orders? Can you add = -v to the end of the compile command: > > >=20 > > >=20 > > > c++ -target x86_64-unknown-freebsd14.0 --sysroot=3D/usr/obj/data/src/= hardenedbsd/amd64.amd64/tmp -B/usr/obj/data/src/hardenedbsd/amd64.amd64/tmp= /usr/bin -fomit-frame-pointer -O2 -pipe -fno-common -DHARDENEDBSD -DNO__SCC= SID -DNO__RCSID -I/data/src/hardenedbsd/lib/libc/include -I/data/src/harden= edbsd/include -I/data/src/hardenedbsd/lib/libc/amd64 -DNLS -ftls-model=3Din= itial-exec -D__DBINTERFACE_PRIVATE -I/data/src/hardenedbsd/contrib/gdtoa -I= /data/src/hardenedbsd/contrib/libc-vis -DINET6 -I/usr/obj/data/src/hardened= bsd/amd64.amd64/lib/libc -I/data/src/hardenedbsd/lib/libc/resolv -D_ACL_PRI= VATE -DPOSIX_MISTAKE -I/data/src/hardenedbsd/lib/libmd -I/data/src/hardened= bsd/lib/libc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/data/src/harden= edbsd/lib/libc/rpc -DWANT_HYPERV -DYP -DNS_CACHING -DSYMBOL_VERSIONING -g -= gz=3Dzlib -mretpoline -fPIC -flto -MD -MF.depend.malloc.o -MTmalloc.o -Wno-= format-zero-length -fstack-protector-strong -Wsystem-headers -Werror -Wall = -Wno-format-y2k -Wno-uninitialized -Wdate-time -Wno-empty-body -Wno-string-= plus-int -Wno-unused-const-variable -Wno-error=3Dunused-but-set-variable -W= no-error=3Darray-parameter -Wno-error=3Ddeprecated-non-prototype -Wno-error= =3Dunused-but-set-parameter -Wno-tautological-compare -Wno-unused-value -Wn= o-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-unuse= d-local-typedef -Wno-address-of-packed-member -Wno-switch -Wno-switch-enum = -Wno-knr-promoted-parameter -Qunused-arguments -I/data/src/hardenedbsd/lib/= libutil -I/data/src/hardenedbsd/lib/msun/amd64 -I/data/src/hardenedbsd/lib/= msun/x86 --include-directory-after /data/src/hardenedbsd/lib/msun/src -DHAR= DENEDBSD -I/data/src/hardenedbsd/contrib/snmalloc/src/snmalloc -std=3Dc++20= -mcx16 -fno-exceptions -fno-rtti -DSNMALLOC_USE_THREAD_CLEANUP -DSNMALLOC_= BOOTSTRAP_ALLOCATOR -DSNMALLOC_JEMALLOC3_EXPERIMENTAL -DSNMALLOC_JEMALLOC_N= ONSTANDARD -DSNMALLOC_PLATFORM_HAS_GETENTROPY -DSNMALLOC_STATIC_LIBRARY_PRE= FIX=3D__ -ftls-model=3Dinitial-exec -DSNMALLOC_CHECK_CLIENT -DSNMALLOC_FAIL= _FAST=3Dfalse -DNDEBUG -g -gz=3Dzlib -mretpoline -flto -Wno-c++11-extension= s -c /data/src/hardenedbsd/lib/libc/stdlib/snmalloc/malloc.cc -o malloc.o > > >=20 > > >=20 > > >=20 > > > That should dump the include paths. It=E2=80=99s possible that the li= bc++ headers are being included in the wrong order with respect to the C pa= ths? > >=20 > > Hey David, > >=20 > > HardenedBSD doesn't have any changes to userland that would cause > > changes in include header paths (and the priorities of them.) > >=20 > > I made sure to `pkg delete -g gcc\* binutils\*` before running another > > buildworld, ending up with the same error. > >=20 > > I tried running your command above. >=20 > He asked that you add a -v to it. Implicitly also: capture > and report the extra text that the -v causes in the output. >=20 > That extra text covers what the sequence of include paths > searched are in the order searched. It shows even the paths > for which nothing is/will-be found for the specific > compile: where all did the compiler look? Any unexpected > places? Any missing places? Any examples of not being in > the required order? >=20 > > Here's the result: > > http://ix.io/4nS9 >=20 > That does not include the extra text that would be generated > by having added the -v requested to that shown command line. > That tet would likely have been before the text that you did > include. >=20 > Did you add the -v option? Was there extra text? Good catch. I missed reading that. Here's the new output: http://ix.io/4nSy Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --utl62kziietstoa5 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmPpVWoACgkQ/y5nonf4 4foQ2A//SHF+npe/6nRSwzTOPVAQjZXH6Xkfvdka6+qH3yVscNPRephxhmpyUEVb WfvaX27T6My4S7gSP4kJX4hltsqeLHszhX7a+MZFN1jORuKfndOPDIcNnkIWq22T 53Pwh49fEMQAUJVTw4NTZKKiHRYWXWCD39u6TLJPMI6c2sY0qfbvPjpy+YprWdso K8SXKCFTREVKg87SK+olgvlD+SgHiyQxTJcmVP2i5BUhnKZipeDkteI5zvMyqRTJ DCMLbjUCH+El6VA2qhBNbdT/0aAvjkcNoRHAioxw1MWO2CEHQ+MjoMT9Wit6ME8N 6MlYHsrtg46RSk7ipV2m8Ks4feiiGsF5qhzh/Ud4PlRV6Feif8N0cQ6xFpFBG4// RpqiKkx8tTXODSzwc0bFekhmL+cX9tC0KXo/QuIiNWX1soznNLK+Ne92mYIzvob2 kSNHJ/rA4ScHy2R+ZvJtnCrqZnF6GhoLbBew9aqLfVqGhOMaNdhOhombncaWcC0B 8zlO2TCY0UiZ6hQMiEywwncOCxSxfe+n2sGWbMHpW/zfDgZ5TDp0YT62Ev/BnrT1 zb0DOjHfQBgEVFikgkVtesldFZy2bJRrl2KECKLeEKxdJaVCMQ2vPj32wDMyV3nF Xpl3C1yJwGYYc8OFqcdwqeq4cx5UUglqQkb2YWPcbPlJ+bFnins= =GCtU -----END PGP SIGNATURE----- --utl62kziietstoa5--