From nobody Fri Nov 14 20:14:57 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4d7T072wYBz6H3fQ for ; Fri, 14 Nov 2025 20:15:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-20.consmr.mail.gq1.yahoo.com (sonic306-20.consmr.mail.gq1.yahoo.com [98.137.68.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4d7T070f5bz3MRh for ; Fri, 14 Nov 2025 20:15:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1763151312; bh=dj9QTynhtX6bANinEoBFcYKbHmRjJ90pY1I64aW/hSQ=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=BAInMCQipA+UUxJzknYSBdHmWW688MJ1efVfhty0HOZzMejMxM3NRakYeJNB7V0tlFT96URhjJd+6JIg9Qfv3CZvhSHmD8LciVk9bGdoLG2vW+Li/4KDeaV1UJs5MngZApsA0DDyQgM5Ef1Dm7TdwjBXDSW5I1ZwUAbex1Lw0VcW0lu4h5N5kH4hqK29wNFWYgmjHrJG7EDIx7vF3F7hthqlR7xOtxFAU25I8NVq1PUaYQAuIqRJ5I0rBh4O3iHl5KQ/uA+4UFR9AtBncDMjcjv1b+KqrKWbGEPkLA+XjUBNXZzdUZNYhZ+D6K5MVUzkggHbLCNJXaMY7LOSMseDCg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1763151312; bh=siuvQKm+Y910O/MxzxIdh3F2UhU3z7wnEqzvtPVJOam=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Juku+8UO81wlPACb3Ibd2ulDINI5bzSiepPtvpRnnGqac2p9eEsfOhMC5lOxPG5u93JlzHZnclhV7ZjWt+QzGwqqOq7yLKqovGHNMEDm4a438WESJvkeRcTtrGtBEN3Z1d7uLaXUgvaJn+NOgvibj1HIzFj5Oa60EBLNG0yXSyFMA59Tw8b93yer+F72ehZvnOTx5MnEZXpYKkS0rpwRl464nbPzC5rgsiTfGapV7vW0w78YjXXUcCl+HyfC/FCIoOmJZ5zTgsc705306HR69MhzTagk0hBLKkqy0Iq5qx1YudWoUR9kxYBVx/Hzql1aGeohmhoxSGq1efmA2GLAZQ== X-YMail-OSG: BaekGIIVM1lGM_c_Pap05HHUl41_7xwpbTm0eLGq48UJ6SxnL_CThXD9bm695Js QgcGIAE.6dGuqmgqt_r5Z3nyOFCeQIs0k5NeMe686NzsFdNzDDeGORLw0W2ROKMnnkJbyWzYUkry Y5mBVkVhIrmew8leIF_UELKHpkLySqag5.olMud2TaqEr9Oqau_z8d2fUEVUl385KqTfwHb7CU5P .Gf0SGhtAVrZ5Klq22hGNwW5fAgIYzAU4wmu5ITngSC8PXXjj7FkEti3sMbjzPQRj1x8dmgq_x3m DVvOahGvgr6nYAocMhewT3thHVhdrdiIfoGIsRNjDvSApEfTW0REJ_uXJ2jsht4ALfD.H19vGbvl I_Bf9JO0CSref.ocgb1DOJsc233neRmb4IHuXC_27tezTW60UvBnsMhuVw3z22FpOrgGJ_ISou6d MXppruZ6gqd9HpBQ_8.S2JAar5jpZFhjMYPWWtzpsxU35AIL1v0Ay_o4ecJvzKDso0N3N4lmHcba lXEUhF44PxJd_Q6sLxMBi4UA48IeKeVV_DBoAB0SErBXGUr3dxVEhwj4Oeagd5lUzmx4XV3gLqAN NQU7AIZ4xaJ6UGtIsOiv8u5VG8jAamqwsyQRk.BgQkdU57dugfL8rthtZssAKwUvCrYEICTiN8YC FPQCd_eZk.bYL6UDswBj7gicnh6nekVGtoPu8NuOgrW_2h8YXyWbfy0Mf.6X0IBKiSsM4pQJ0vW7 9LC.D4YA7gd4SfZC2huo3skWPGbxs4YUQgrFAWDajCwfXzseizIU2oAs4bwccYicnGRoAZgxulN4 Ko.kBTwKM0CeYsEjWxNtBvNn5Aj_7L25TEuBQQCZK.iesPU1selmaaOv_.qsHG9B79hXEQVWB5em l0R.l5AgN1K2sllAPTfsHpksVTiOBf8vkFUxbsgYCBiKXjA0WB0xBVM5awzWT5eN5DDUaaMiE5pS 0i5IO5j8LPGH4PqAurVihJ3B2dtoXxQN4qQQno1xaAreYfQD57QUewuZbSmRWLdqq5k46m.GnXVS 6zfyJXmdNQLjxd1q6DBq7a42eIwFUIquS4ANfazDU966c1bawV.KQkUmYMO7INKm9h2pBojkRROm FaQl2BkpPWDHyUmYJe_FRzDoRcMUJnL5Y3Al7aEPqcEWrmwN5z.dfGYWsgzO5DMKKO5wh7Lchveu 1kOvjaR0xpwVBoVptLmYdDUm.Xy_mupRYKiQ5iPOeTZs9rDk0sWvo_vQy5VNyMDWnvBe0_hw7Ity KzWS3IEiybQRLcep8CJklIBFFNIvJjcGetNhr8L8rTXVrQ5aKmaVI_TtbrVTBnmd7mloKbl_FAVJ 4JOfrRIx87Vd4qtC.b7S1F98IA5aj8CM3jM9BnuKUxvmr7M1rvQnT9VRUQ96v5fojbY1XliGGxQ4 QtUukzzpZIbPaOl40JHEjiJ8TKeo1NERPBBK.4hPHLI9hz0RCkZ5Jcp487AMXp65a3I5j2zFgEcN CdLnjkCd2RGzg8ZjpYQHpPT2hMsvozwxlnQMpgDiJWnlWEzMva0AmYQG6ZPYSTcMD6K_.mUWwP5o fdtBpsN3OgDVTY6f1PSx0u3dzQddShiNY0fnn03EsaoBT.Q62hNeO0d4MrqvWhwn5EJQKHrJG5sA d1yLsxN88SvINcYRDhlkUV608wlzsD4Cj3iUmjoSmOs3dN.QXtG_nk2Z5OUIpAe2lUncPfRFZMPt jMwmGzB4WHnWvNNgcqgbX9SSPgTXHSzp017qdDRAnAZpvs0lOa_59QRFyG36smhguP8R31TB_5w. eg8tFzNd0PfzvzUHL_ywbc1Y7.OtkVLdbX_T5lUgCbcJkihpRMezAfbwMRqLbfNh9cOAH6hfVQI1 HMfMbPYsfUqBB82rt6gGx8mbWUXbfw3Wql3ZeQcbBWdO2SXjQFpYN.vr4TuUOGaCxCfZyDwC8J8n _vv9rhfjTI.a0BgxSZqamNqvk4vZBjB8HswkAsir8NgQuLGc2eauimRPO6R6JSLvKtuR5hZam2jS JimYV6TbEjwijeP48kqCVmmV71GNos08oPeq8nRcrIa7qffIQHobkiYeySyxmvuB8ugQ.3kOUq0r WOpktWVWkia1dS8.3TRtbCKyyeQLTv53e1GBpNgGUp48YZ8p0905k_Th5TXE0t8_.Opdgy_c5Vcz WlHk9Lq8JjuhReCXSFT2.fYgtkx4vxcY0ZyFJGevWmSZaT7V3fHjLrDvf2A2gE_JEHQw139vdK7F MyLj7xZMA4zTbjczxRTNuzdMv12dCsim73oCETqY6pC7zSsDyLnkz4g3.WDMfQ3tci9N40xhQICK bX1zc76A- X-Sonic-MF: X-Sonic-ID: e26c631e-ec12-4bc3-99d4-27f252988368 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Fri, 14 Nov 2025 20:15:12 +0000 Received: by hermes--production-gq1-76c986f798-j5tsw (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 1b5eaa041db4b9b6af84b555f8544876; Fri, 14 Nov 2025 20:15:08 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Subject: Re: Still seeing Failed assertion: "p[i] == 0" on armv7 buildworld From: Mark Millard In-Reply-To: Date: Fri, 14 Nov 2025 12:14:57 -0800 Cc: Carl Shapiro , Ronald Klop , freebsd-arm@freebsd.org, freebsd-current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <08A61538-82A4-4CB0-8610-FF61CDA05EE8@yahoo.com> References: <4957be52-e57f-4f5f-9626-d0f706480fe1@FreeBSD.org> <87ldkai9lu.fsf@panix.com> <877bvthymv.fsf@panix.com> To: bob prohaska X-Mailer: Apple Mail (2.3826.700.81) X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4d7T070f5bz3MRh On Nov 14, 2025, at 11:25, bob prohaska wrote: > On Fri, Nov 14, 2025 at 09:16:57AM -0800, Mark Millard wrote: >> On Nov 14, 2025, at 07:25, bob prohaska wrote: >>=20 >>> On Thu, Nov 13, 2025 at 11:16:56PM -0800, Carl Shapiro wrote: >>>> bob prohaska writes: >>>>=20 >>>>> All the assertion failures I've seen have been in the clang = libraries during >>>>> buildworld. They appear to happen in a variety of cases, indicated = by the=20 >>>>> different .sh and .cpp filenames found in the files under >>>>> http://www.zefox.net/~fbsd/assertion_failure/ >>>>=20 >>>> Do you have the stdout and stderr of the build somewhere in there = as >>>> well? The make(1) invocation in the readme file shows its output = being >>>> redirected to a file. >>>=20 >>> Those files have been overwritten by restarting the buildworld = sessions. >>> They tend to be large and diffcult to synchronize with the .cpp and = .sh >>> files generated by the crash. It could be done if it's useful. >>>=20 >>>>=20 >>>> The assert you mentioned in the subject of your e-mail message, = which I >>>> also saw in the readme file, could come from jemalloc. See these = lines >>>> of code for the context >>>>=20 >>>> = https://github.com/facebook/jemalloc/blob/dev/src/extent.c#L805-L814 >>>>=20 >>>> That assertion will be tripped when jemalloc sees non-zero memory = that >>>> it expects to be zeroed. See for example >>>>=20 >>>> https://github.com/facebook/jemalloc/blob/dev/src/pages.c#L55-L106 >>>>=20 >>>> Looking at the code, my hypothesis would be that jemalloc thinks = it's >>>> committing memory for the first time but the memory is coming back = with >>>> non-zero data. >>>>=20 >>>> Just curious, but is over-commit enabled on your system? Here is = the >>>> signal jemalloc is using to check >>>>=20 >>>> https://github.com/facebook/jemalloc/blob/dev/src/pages.c#L729-L737 >>>>=20 >>>=20 >>> Sysctl -a reports in part: >>> # sysctl -a | grep -i overcommit >>> sysctl: S_vmtotal 48 !=3D 88 >>=20 >> The s_vmtotal line above is from what >>=20 >> sysctl vm.vmtotal >>=20 >> would report: output for >>=20 >> "System wide totals computed every five seconds". >>=20 >> That S_vmtotal line reported is a internal warning from >> sysctl. The 88 is correct and is sizeof(struct vmtotal) >> from sys/sys/vmmeter.h : >>=20 >> (kgdb) ptype /o *(struct vmtotal*)0 >> /* offset | size */ type =3D struct vmtotal { >> /* 0 | 8 */ uint64_t t_vm; >> /* 8 | 8 */ uint64_t t_avm; >> /* 16 | 8 */ uint64_t t_rm; >> /* 24 | 8 */ uint64_t t_arm; >> /* 32 | 8 */ uint64_t t_vmshr; >> /* 40 | 8 */ uint64_t t_avmshr; >> /* 48 | 8 */ uint64_t t_rmshr; >> /* 56 | 8 */ uint64_t t_armshr; >> /* 64 | 8 */ uint64_t t_free; >> /* 72 | 2 */ int16_t t_rq; >> /* 74 | 2 */ int16_t t_dw; >> /* 76 | 2 */ int16_t t_pw; >> /* 78 | 2 */ int16_t t_sl; >> /* 80 | 2 */ int16_t t_sw; >> /* 82 | 6 */ uint16_t t_pad[3]; >>=20 >> /* total size (bytes): 88 */ >> } >>=20 >> The 48 is wrong for what the internal sysctl(. . .) >> returned. The message also indicates that the >> normal assocaited output was not generated for >> vm.vmtotal . >>=20 >> I do not know if the error is somehow associated with >> your overlarge swap space (if you still have that). >> In my context "sysctl vm.vmtotal" and "sysctl -a" >> are working normally. >=20 > The 48 is likely related to having excess swap space. > On a machine with 1.77 GB swap the command reports > root@www:/usr/src # sysctl -a | grep -i overcommit > vm.overcommit: 0 It still indicates some invalid internal state in your system. I suggest avoiding being in that status until after your environment no longer has its 2 other problems (hang-ups and jemalloc assertion failures). > I don't think it's related to the assertion failure, > since that host experiences assertion failures as often > as hosts with excess swap space. I recommend avoiding having any reported corruptions active during the search for solutions to your 2 problems: Avoid guessing about interactions of oddities. >>> vm.overcommit: 0 >>=20 >> "man 7 tuning" reports about vm.overcommit : >>=20 >> The vm.overcommit sysctl defines the overcommit behaviour of the = vm >> subsystem. The virtual memory system always does accounting of = the swap >> space reservation, both total for system and per-user. = Corresponding >> values are available through sysctl vm.swap_total, that gives the = total >> bytes available for swapping, and vm.swap_reserved, that gives = number of >> bytes that may be needed to back all currently allocated = anonymous >> memory. >>=20 >> Setting bit 0 of the vm.overcommit sysctl causes the virtual = memory >> system to return failure to the process when allocation of memory = causes >> vm.swap_reserved to exceed vm.swap_total. Bit 1 of the sysctl = enforces >> RLIMIT_SWAP limit (see getrlimit(2)). Root is exempt from this = limit. >> Bit 2 allows to count most of the physical memory as allocatable, = except >> wired and free reserved pages (accounted by = vm.stats.vm.v_free_target and >> vm.stats.vm.v_wire_count sysctls, respectively). >>=20 >>> #=20 >>> It's unclear if this implies yes or no, or even is the correct test. >=20 > I remain uncertain if overcommit is on or off 8-( It seems like = overcommit > limits are intended to keep one user from exhausting swap on a = multiuser > host. Not my situation, if that's the case. =20 It has nothing to do with single-uaer vs. multi-user in general but only in part (i.e, optionally). Also, some of the bit mask values are not really about overcommit but are about somewhat related limitations. More than one of the options can be enabled at the same time but normally none are enabled. A description of overcommit: vm.swap_total < vm.swap_reserved My notes below are in terms of applying a bit mask to pick out a bit and have the result be !=3D 0u (set) vs. =3D=3D 0u (unset). The mask 0x1u being set would lead to rejection of attempts to have: vm.swap_total < vm.swap_reserved You do not have 0x1u set so such rejection is not enabled. The mask 0x2u involves: getrlimit(RLIMIT_SWAP, struct rlimit* cur_and_max) Quoting: RLIMIT_SWAP The maximum size (in bytes) of the swap space that = may be reserved or used by all of this user id's = processes. This limit is enforced only if bit 1 of the = vm.overcommit sysctl is set. Please see tuning(7) for a complete description of this sysctl. So this is a user's-processes-specific limit. You do not have 0x2u set so RLIMIT_SWAP use is not enabled. (Not really overcommit.) The mask 0x4u set might lead to more RAM being considered as allocatable. You do not have 0x4u set so you have normal allocatable physical memory classification in use. (Not really overcommit.) > It can only be said that it's probably whatever is default for = -current. > The sysctl command above was run as root, as is buildworld. The default is always 0x0u: none of the 3 options being enabled. =3D=3D=3D Mark Millard marklmi at yahoo.com