From nobody Sun Feb 12 20:01:22 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PFJJf0gN6z3rBwj for ; Sun, 12 Feb 2023 20:01:38 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PFJJc05dJz43Qf for ; Sun, 12 Feb 2023 20:01:35 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=JIfhV2oV; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1676232094; bh=suS1YhdBrEFsreozH+4w7JZ9WJIFXkceKwnreqtF9jY=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=JIfhV2oVDKex7mKIgx38Ps+qi3EpD2uplqNzf6Ly7EaoQ93vfRd26c/SLI0NwCI0NltjNedcE/zaN73DsGIyrwvnHpwYM32VPHxAJ4KJgEzb5d2O+syLp90cMviFCNwHpmXbsX90UN8m2PgPw6bUJ7Oo9QEr23FiS4KOvR7Q6JhQzPLxxVhojVjrz+2WJ45QS+GewcVL9ESwX7LLSnQHqlGCU6QpSf8C+kIRNV7hK7yoKr/78RWwxlKGxMrE56UqXWXKxv1vuOB4OYyroxlcTu+AMAU8hAXzqcuicSyK+B/YQs+uA6mn0hnTusorbhioCOQiIuX5ndvI7rIfGagO3Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1676232094; bh=Cb27un9AZnvHjtHBmvJfQZ0c7u7Deud71hQ/5Eq6tXw=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=X8QcqZsiNF7bGTo3jEDKNye5GRP3a9vyXPYEwX2/z+UUvJcg0ZBXwJfHAR5h2QkQtk6bsjvJHX5EEXGfeaMlt72OUQRS+zmkarxFEIM3nWpPbSh7Py4W9FYWxPZUsttdsJzKSCCt0mKsRYYZjQUbGXBQqgQ78f4m9++Ow+Tjt8X8E8UcevGrxtzrj1pG00VQFt7kjnggZ7hB6qQ8SWwOdX1pLbicvaq1EINLaMKBeprMo0AU/67ZXRvVxkAYGzKuyT+2qoQGYM4g6R0Iq/RnoH4cigkFfa/JviAA1E5i/KvdQKp9jr2Et47BYxPlm+1os8rXHd3aMxS+eN39SofVLg== X-YMail-OSG: XomcE7UVM1nvIG.qAXlwuEdVlsIMaPqnIirELvxWAdHrrZv_dI0CChNCwTQiSPl .kuoBOC_VlK8EVkqyapuuAj3tkW1l5hecQfF.TIF4ZS6ZEujNAq0.EldUmhSVz6BTYPdzBLFdIAM i7UAdgXlrS0HpSs7WtcAfbWIEmMUydzZZOvpVwCIwm0haZG.zjHMB_YfRE6sTp9678KORZAUh53O Pl5Vf.86xz9uXbnSuYa4avJI6ZTCyF1WhlWvcAw34qFwTalYF0eajORvluIszreEfU8iSaxbBirD Sg7oMPk_0xPJP9uRPZr1gNR2q1Y1BR9ugRhVnwWgCVBeE01j7XNNsJNjnizuOdw8VfFWHf9mNdQ9 3WtwK.x2v1cgcf5sWgkZTXdISlrZ_pd.6fjD7oBzoTAYN4hOMSNVnIJ8gohzep_jpJi_4HfRfVTZ ThhRWTUVpMLfAwq5OYCLOB..8l5W2tNCWAqEwHIGzAmzS2YO49CYKtBU6ea4atgDpWqUeaSZ2Gt9 _YEb8XOrlkGu.1D.UT7ZNv5ZrsePaJ3Nfj62w6G9AkpS_ykgHzuCNxx.Y06szhlVM9VvRBgLjaXv GEZS7IN0RTfbDjo.xxN3sm9_3150x7zRc1gxkQWmhcESa_HW7A33.bUEADim7XNKmmyOHVOCE75I x8316YdUVUht7GKRy_ph7sPZJ0uaJMhD1Y5OGF6HWr.h4J4qVdU8DV68PVOttVdjGBsTJnL5d4vr 0yDStncq6X_gX_NmIUjbcA7ayscYmgwUsuAlaTR7bsbHD64Vw2XyjS548Fg6wU6s0V4Gl.Z6Bq2l VwpsXvtoSpV9XIQfk0_IcwCLt0omg_.dQz_DuqelTvJO.mjMY6mmR_7Tbk951Xi99iLpOQlmEWOY fAloQLKg37N_wStyojfNBjx315YfJ2MLmv_1IfT9qahdma6rLywLiAuhFzO1RpJ_GmuTbAL5BrCc .4a17wD4zFgBt5r4l2rqVmqX1v7BIBWSmtvnCSZslyeZKxHLeeczOXbC8PEvt4eFEIIBfKnlYg_u soy_XUlyM4LjYWCoSv2bzC0QwpthRmWXLONqQmoVGJU9OAvN0uRhuaIwm7Uq2vIcZbUmBehvCbTM DdMOw0dFBg7YdMAxsAsdRAaWnMVHsSF9p3lG0P46dVSnlO5ykMOir2Yo0mq2Cc2eSlJCddPpx8qq BwMAQ1arrAAyHqKtTkSMl9NR8XN0GUCiUdRolLs1OkgZsQt17FNPBHzXELbzJJ_1Fghg93yCgRaJ 8JM0VWuWuS9PKMojnZMg5rmPAWxAAw3pKghtZ3OMtPMPZqWK3kmnmu4BrUq7wcuz2QVeYTT8dSJ8 RhWKYpalpXf3xmATk673N.5uZ_b_KIoYCXnnxAVpgtIfolFyQs3lEVImNQ7T6tjGZa03qXlVRHuP .uCu1NMrMpcuqQRlcwljZd_LKnH8Mc.YXHzsrUz4LPBSxhiuRIluZAJa_2Dx_4aflcx0F6WBiLfp LHAAsuN7R2fYtZh6Y_Rq_iuJAC3amZ4eausxjzm1NpI_RzcssUk6vC5gq.m5lAsGa.IM1L3Y2Iu1 iM7D3cDbspBmbr6I4UQGV2ue6CMkHVhNLtHIvuXf4bblcH.2UHJi0r6x06_jyLd755IebStG7wkO yJ4kYXG0BMbiwyXUHHvlULeVBDqtQ36ZWL1X5CvARCegeD1Ak4Pmlrq7I9FiIV2XivIVK9ALzjOU UFUF2eDFxrqH6PPlpJ5g2eW15qt0HRtHGitzEJouZpSlsRWTAiWTMo_W.pTpl0vmW2ZDcjaQ7ZEm w4NdU8ZeJznk.PcqebqRE5EQeDXPwK2i89bYDkmzzaNUgt0_nSq9CNwvPZiA4KhlmzDR1KseyXhe uD4du64Q8xhA1_XF4Ur4E_7zUwMmTSJaBaihhiz7I_Y.9ZogmEK5JmCPdCcNtO0.YYlU_2kjuB4T i2kKHDTI1qw0asJ8bsqDKPiQKkwWWDS0IUsGy_CbopFYIDR_fVugnEZWe1UUGoRGJSbkZ3jYQiYY 9SoejhBGF3UhYzK5KmyAeJkQzcQrx.b7gPZnaVV7A2D_mELdheNCTnevZ.CgkUQ36TwNuK1r3ysZ B7soCVRnR9K.OsYPBWoEXI9pZNYUpoB7Xz5cae.Dh6X7hHKTPXXKLXwJ.gu.P_pLZZaDQLTYVnLR NWxZtYrn2xNg4Af0A1p8Dbm0BtG0kkA3lQNlWsssvJGJ862YdR1NeHC.zcVn.cDP9YobZ1K8sGEA ksZljSoLPjFjQnDXS_LjPQY8dOu2AoACe6uKxX4ZLyCMgg_uTdQpy3v7gIkqDcFcIGuK7rp2UjM9 Sq3qIwKD5 X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Sun, 12 Feb 2023 20:01:34 +0000 Received: by hermes--production-gq1-655ddccc9-j6kw5 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID b82b2dfb47e95f30041923d27d234d9e; Sun, 12 Feb 2023 20:01:33 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: CFT: snmalloc as libc malloc Message-Id: Date: Sun, 12 Feb 2023 12:01:22 -0800 Cc: David Chisnall To: shawn.webb@hardenedbsd.org, FreeBSD Hackers X-Mailer: Apple Mail (2.3731.300.101.1.3) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org] X-Rspamd-Queue-Id: 4PFJJc05dJz43Qf X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Shawn Webb wrote on Date: Sun, 12 Feb 2023 19:44:24 UTC : > On Sat, Feb 11, 2023 at 05:10:02PM +0000, David Chisnall wrote: > > On 10 Feb 2023, at 16:23, Shawn Webb = wrote: > > >=20 > > > So I took a little bit of a different approach, which should = provide > > > the same end result as your submodule approach. Note that I'm = doing > > > this in HardenedBSD 14-CURRENT (the hardened/current/master = branch). > > >=20 > > > 1. git cherry-pick -xs a5c83c69817d03943b8be982dd815c7e263d1a83 > > > 2. git rm -f .gitmodules contrib/snmalloc > > > 3. git commit > > > 4. git subtree add -P contrib/snmalloc \ > > > git@github.com:microsoft/snmalloc.git main > > >=20 > > > I believe this should leave me with a tree that populates > > > contrib/snmalloc and pulls in your non-contrib/ changes, leading = me to > > > end up in the same end state as your submodule approach. > > >=20 > > > I am seeing some build errors. I've uploaded a WITHOUT_CLEAN=3Dyes = log > > > to: > > >=20 > > > https://hardenedbsd.org/~shawn/2023-02-10_snmalloc-01.log.txt > > >=20 > > > Note that this is with llvm 15.0.7 that just landed in FreeBSD = main. > >=20 > > The error is a bit confusing, because nullptr_t has been in libc++ = since C++11. Does HardenedBSD change anything in include orders? Can you = add -v to the end of the compile command: > >=20 > >=20 > > c++ -target x86_64-unknown-freebsd14.0 = --sysroot=3D/usr/obj/data/src/hardenedbsd/amd64.amd64/tmp = -B/usr/obj/data/src/hardenedbsd/amd64.amd64/tmp/usr/bin = -fomit-frame-pointer -O2 -pipe -fno-common -DHARDENEDBSD -DNO__SCCSID = -DNO__RCSID -I/data/src/hardenedbsd/lib/libc/include = -I/data/src/hardenedbsd/include -I/data/src/hardenedbsd/lib/libc/amd64 = -DNLS -ftls-model=3Dinitial-exec -D__DBINTERFACE_PRIVATE = -I/data/src/hardenedbsd/contrib/gdtoa = -I/data/src/hardenedbsd/contrib/libc-vis -DINET6 = -I/usr/obj/data/src/hardenedbsd/amd64.amd64/lib/libc = -I/data/src/hardenedbsd/lib/libc/resolv -D_ACL_PRIVATE -DPOSIX_MISTAKE = -I/data/src/hardenedbsd/lib/libmd = -I/data/src/hardenedbsd/lib/libc/locale -DBROKEN_DES -DPORTMAP = -DDES_BUILTIN -I/data/src/hardenedbsd/lib/libc/rpc -DWANT_HYPERV -DYP = -DNS_CACHING -DSYMBOL_VERSIONING -g -gz=3Dzlib -mretpoline -fPIC -flto = -MD -MF.depend.malloc.o -MTmalloc.o -Wno-format-zero-length = -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k = -Wno-uninitialized -Wdate-time -Wno-empty-body -Wno-string-plus-int = -Wno-unused-const-variable -Wno-error=3Dunused-but-set-variable = -Wno-error=3Darray-parameter -Wno-error=3Ddeprecated-non-prototype = -Wno-error=3Dunused-but-set-parameter -Wno-tautological-compare = -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function = -Wno-enum-conversion -Wno-unused-local-typedef = -Wno-address-of-packed-member -Wno-switch -Wno-switch-enum = -Wno-knr-promoted-parameter -Qunused-arguments = -I/data/src/hardenedbsd/lib/libutil = -I/data/src/hardenedbsd/lib/msun/amd64 = -I/data/src/hardenedbsd/lib/msun/x86 --include-directory-after = /data/src/hardenedbsd/lib/msun/src -DHARDENEDBSD = -I/data/src/hardenedbsd/contrib/snmalloc/src/snmalloc -std=3Dc++20 = -mcx16 -fno-exceptions -fno-rtti -DSNMALLOC_USE_THREAD_CLEANUP = -DSNMALLOC_BOOTSTRAP_ALLOCATOR -DSNMALLOC_JEMALLOC3_EXPERIMENTAL = -DSNMALLOC_JEMALLOC_NONSTANDARD -DSNMALLOC_PLATFORM_HAS_GETENTROPY = -DSNMALLOC_STATIC_LIBRARY_PREFIX=3D__ -ftls-model=3Dinitial-exec = -DSNMALLOC_CHECK_CLIENT -DSNMALLOC_FAIL_FAST=3Dfalse -DNDEBUG -g = -gz=3Dzlib -mretpoline -flto -Wno-c++11-extensions -c = /data/src/hardenedbsd/lib/libc/stdlib/snmalloc/malloc.cc -o malloc.o > >=20 > >=20 > >=20 > > That should dump the include paths. It=E2=80=99s possible that the = libc++ headers are being included in the wrong order with respect to the = C paths? >=20 > Hey David, >=20 > HardenedBSD doesn't have any changes to userland that would cause > changes in include header paths (and the priorities of them.) >=20 > I made sure to `pkg delete -g gcc\* binutils\*` before running another > buildworld, ending up with the same error. >=20 > I tried running your command above. He asked that you add a -v to it. Implicitly also: capture and report the extra text that the -v causes in the output. That extra text covers what the sequence of include paths searched are in the order searched. It shows even the paths for which nothing is/will-be found for the specific compile: where all did the compiler look? Any unexpected places? Any missing places? Any examples of not being in the required order? > Here's the result: > http://ix.io/4nS9 That does not include the extra text that would be generated by having added the -v requested to that shown command line. That tet would likely have been before the text that you did include. Did you add the -v option? Was there extra text? > I think Mark might be on the right track with regards to llvm 15. I'll > try importing the snmalloc patch (via the same subtree method) to a > pre-llvm15 import branch and report back. It might be a few days > before I get you a result. >=20 > BTW, I appreciate the work. One long-term goal I've had for > HardenedBSD is to be able to switch between multiple heap > implementations at buildworld-time. I think it would be nifty for > users to have a choice of jemalloc, snmalloc, and GrapheneOS' > hardened_malloc. =3D=3D=3D Mark Millard marklmi at yahoo.com