From nobody Fri Nov 14 17:16:57 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4d7P2l4p7Qz6GnJL for ; Fri, 14 Nov 2025 17:17:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-54.consmr.mail.gq1.yahoo.com (sonic315-54.consmr.mail.gq1.yahoo.com [98.137.65.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4d7P2l3Mczz3vjT for ; Fri, 14 Nov 2025 17:17:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1763140633; bh=Me8+ff5Lk+E7Ksxauz/VjF2Rf3D4utgpQWA0NbDIY0c=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=rS2SFo3BMQmUfCHjG8oTwz74eBabEmH1OFq5jNw7CMC/2Y8kI0cGcuYeCpD79qfX0aoWBUp0AcgGiUVgW+rgcC/L3kAqnrtTVUkauJGZolsynNENWH8dOKM10ME4xEpzVHe5Sb5SHUUvyiRaHNo5Gc9aA1MaVKhEJxfkl47Fmbgcm9nhSXwCDE1kVqshg4c/8bEXstj+iCE1FIhv3TFbMwUhgP5uVUCqUx/FdTXKkAtLLrGw50SXrABbdnIHMvXoM3bpdGh0vKZvuzOOfvUftsIek7Coaz+xAqhVspplvdaOjJ83jK1cJ4L0Od601TWji8Q1QNuidImXzeD2pm58Eg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1763140633; bh=HMgk9FBjJ8RXOiZ+kgdGkwdANCMTnD0b68nBodv1VUK=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=cQxqgiBl0CgxZrJloBtXOPP8HqZJK9Gah7UiI6l6vT84mLNaWaTYtRhqV1sH0ZA/JqyA8mk/t/KlF9yrBaMBzuvUxIYpF2bk5qZJ7zp0x92tBDVMaiMhLrc0jPKke6pp6SucBaM7bHXOcqwlx+aMx5aon2DCXsLsuipMgbenwAt6rWTyh9P9w6vfq2o5srylERjmzD0iqladqME5/UTPcbewH3zb7p+x2MWJlbDqm6jEmsgIl2dgO89WXxsluJr/VLDG1DySV0Wv7iZgWwJBgxeQcfUI37MifnCbtwuK/uIqXwiqA9wRes3l+R+5+kINZwyvO2hVKUahPiLXUoJnmw== X-YMail-OSG: qxG7x_cVM1mtVfzIFfK8zgccaonwCJHGyct328HMmXlfbr49vS17q7jBhigtQki gCz6NaMwphI6k99SkJwaYWbvuCu1kLJY9KqfPd7LtkjHz7jcjUVpuz3_VtA_oo.QK79pkE4fm0ws VMJXrOsdKsrcYabTN6xI74vzRJjCD3MhTkbr1r9tFZ9GUuTzCSeSQ.KCLiwtKA3ptt_v0JY_NRPe l9qZkXQyD31FHk9_Nqz9mlGLg3789oejedx1fET2iuhOLZHCIxz.Uo11yNysTkf74H7U.b33ABrk cEUTvFkNuB4HFpxoiyVyo2g__RwbmK9BaIy5SZhBEmo7tF19O.c.bEsy8plLD3vswt1YYzmET6Kf quzel9puYLVr407JFBbMyFPNdjcn9JYjrkcysBOjT1437dLzpuzpioosYippxGUsn8vOzZpQNbqw h.jjN97xDbQwpT59eAtlBniLRGUxbDzuhJGHR69rcRyhQm0J8yn_Rz_Q_2lGGXE_M1XBWp0VTY8G vLFSDJppTmKWBB8MgIHwd2tOrRCFHfszSXQV0g.gqM34abFRGpBX54R3B_.de615BA5gLOllVLqw YXItNMyFWlB9KmCoBgZXZ3xtYZxjItA1LbcDytnOi3dQ9gKZXqipiPLKwWF3lq5Lkim3ZS6JGqry x98AX7J1ruUrTXF29PpbS6G4Pv9.HCjNz.r5Wg4L7Doj421WhvK8l83xzh8snPcirVzSvzVi7F8U a2H_rhXl2wAivDFlxY.M82i8K2xPtM4eMnnUhJaUD_KgSwCn6vfZ6FiMHdqp1vbYtv_vQUO.etIW LUTzEaELEJQUDOSBli3BBBEwtaJhR5qZijW7Mnp1kH_biUhJWxRgyF3q__46pv4.qUZlhO5migH5 ZGgmTUiLlRvVM3rXdCtyHzVCUVAzopiJ8Rd7idJ13cKAQkkWlEdW2ovIOMgdHyzd_0LKn96EH5HD eXyOZRBWiZKrCjoSoDo_kb034T5QF3DAyDW.OEXmMZpoZiZA__qAuXCmYLybBIcrrreHVgRp1FVI aG04obJKz5ODC6gM_h.7p8G40aLCn17a84n1It5KCrv99eHarRGjDoG8XATLHd.6C9kCocuMapaY EUCcus7jeiuPNmISlgx.fCHOF.yY1rhmIdfFSu.btors8uJhQkjdU9HvGnNoigJzSuPKMFZShE4L zMwF2FrNuC_0ARRMcQiYQBUuvylafnsFMRC10d1mxerFZWRJtx7hQkb6tkFQm_4iCL9DEBoNbMft WgPB0tItqnzYkpSo_JRnp60oRxHv79Xz8tQ3.9AO4e.LKlnXu4n9xKcgfX6ybmHl7zASOTxHHy9O XCmFBOXWdsxSXBwLprFXd1LHk2mEKl8yHN8dTWrMPqgOvyOKY_klCOepDHOzwmy.SGMb21Ym_ET1 994T2dd3KfJbAt0BfPHPIBgGo88fWTpSl25mA0M3ZJ5r9Nj9U3PfYkoj8rG.A1ev5AldSaN6JGWI ekbGTbv9UINfaL2_mCNmg4_QnpSeP6wAzRsTVNDje.eZkn9pw1HRatoYbcfD32cE8OECBIYTe9IG w4MH4takA_aNARkI1rnvRof3B7FFc8iJ05MBYxk93kM_VhcYgyFrl.SVHg5JgKKHsVXh3LzFZhx3 VIVqAb9Q6rUhAqcnza78XaOZdGb2HqWvAGIFwkgcLN1K54KwkdFBxZdhUu.trs6uw20dGH0kor.v mvgdng7dW1JSquSr5P5e1O9fq7G9KlgDPCAeg8pWApoBwaVLTJlvfeOH2moxDtKViYRR4sVPuccq rJie3p.KGzpUsaBSIwhH2_.eZiAFeQQx1jFGaYBjapdRKRDBOeB.1WePxfYP8cATzgl39ND4UnlY CA3CZCAV3b1znCF0ZsIwz6mebXo6rj8GJq4pZIsB4btEXvqfOLACyT.s2cgJN6aDlQzz41KBvACa YLjVaUFS.D1gmWfrLvC4xuAbLeR4AwcOf0IkkIa5i.eKaEluUUWfKluWeh1U_5khwJ23SqMJEbgf 1hWiMSjezi1a.PEz1z8cqtrgvCJBv7nBJISjUqausLTc62s_L5O15hEzjNkHXPHrOjdbNnaFQ2uJ 2BnhWgYWUIN8.c7ZJj5BrrLipKXnssctbcjR_ViiHVeZgYvwD8zAs6B39tkz_LUzxxOBF2L9.hVU H2T9hgO94TPh80.5ajGlWbnRkcihloZSlYhiLGS7k09Hs15mDStO7B1eWlFtiuSuDor3GNr7F2jA poRp7FZ9S902qh9ze6CblFXm40gV9h0AO0jU- X-Sonic-MF: X-Sonic-ID: 4f8cafc9-6ec7-43bf-9dc4-a5321b6aec1e Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Fri, 14 Nov 2025 17:17:13 +0000 Received: by hermes--production-gq1-76c986f798-pjhrg (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 08cb25d39aa5967c89a2789bc8399bee; Fri, 14 Nov 2025 17:17:07 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Subject: Re: Still seeing Failed assertion: "p[i] == 0" on armv7 buildworld From: Mark Millard In-Reply-To: Date: Fri, 14 Nov 2025 09:16:57 -0800 Cc: Carl Shapiro , Ronald Klop , freebsd-arm@freebsd.org, freebsd-current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <4957be52-e57f-4f5f-9626-d0f706480fe1@FreeBSD.org> <87ldkai9lu.fsf@panix.com> <877bvthymv.fsf@panix.com> To: bob prohaska X-Mailer: Apple Mail (2.3826.700.81) X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4d7P2l3Mczz3vjT On Nov 14, 2025, at 07:25, bob prohaska wrote: > On Thu, Nov 13, 2025 at 11:16:56PM -0800, Carl Shapiro wrote: >> bob prohaska writes: >>=20 >>> All the assertion failures I've seen have been in the clang = libraries during >>> buildworld. They appear to happen in a variety of cases, indicated = by the=20 >>> different .sh and .cpp filenames found in the files under >>> http://www.zefox.net/~fbsd/assertion_failure/ >>=20 >> Do you have the stdout and stderr of the build somewhere in there as >> well? The make(1) invocation in the readme file shows its output = being >> redirected to a file. >=20 > Those files have been overwritten by restarting the buildworld = sessions. > They tend to be large and diffcult to synchronize with the .cpp and = .sh > files generated by the crash. It could be done if it's useful. >=20 >>=20 >> The assert you mentioned in the subject of your e-mail message, which = I >> also saw in the readme file, could come from jemalloc. See these = lines >> of code for the context >>=20 >> https://github.com/facebook/jemalloc/blob/dev/src/extent.c#L805-L814 >>=20 >> That assertion will be tripped when jemalloc sees non-zero memory = that >> it expects to be zeroed. See for example >>=20 >> https://github.com/facebook/jemalloc/blob/dev/src/pages.c#L55-L106 >>=20 >> Looking at the code, my hypothesis would be that jemalloc thinks it's >> committing memory for the first time but the memory is coming back = with >> non-zero data. >>=20 >> Just curious, but is over-commit enabled on your system? Here is the >> signal jemalloc is using to check >>=20 >> https://github.com/facebook/jemalloc/blob/dev/src/pages.c#L729-L737 >>=20 >=20 > Sysctl -a reports in part: > # sysctl -a | grep -i overcommit > sysctl: S_vmtotal 48 !=3D 88 The s_vmtotal line above is from what sysctl vm.vmtotal would report: output for "System wide totals computed every five seconds". That S_vmtotal line reported is a internal warning from sysctl. The 88 is correct and is sizeof(struct vmtotal) from sys/sys/vmmeter.h : (kgdb) ptype /o *(struct vmtotal*)0 /* offset | size */ type =3D struct vmtotal { /* 0 | 8 */ uint64_t t_vm; /* 8 | 8 */ uint64_t t_avm; /* 16 | 8 */ uint64_t t_rm; /* 24 | 8 */ uint64_t t_arm; /* 32 | 8 */ uint64_t t_vmshr; /* 40 | 8 */ uint64_t t_avmshr; /* 48 | 8 */ uint64_t t_rmshr; /* 56 | 8 */ uint64_t t_armshr; /* 64 | 8 */ uint64_t t_free; /* 72 | 2 */ int16_t t_rq; /* 74 | 2 */ int16_t t_dw; /* 76 | 2 */ int16_t t_pw; /* 78 | 2 */ int16_t t_sl; /* 80 | 2 */ int16_t t_sw; /* 82 | 6 */ uint16_t t_pad[3]; /* total size (bytes): 88 */ } The 48 is wrong for what the internal sysctl(. . .) returned. The message also indicates that the normal assocaited output was not generated for vm.vmtotal . I do not know if the error is somehow associated with your overlarge swap space (if you still have that). In my context "sysctl vm.vmtotal" and "sysctl -a" are working normally. > vm.overcommit: 0 "man 7 tuning" reports about vm.overcommit : The vm.overcommit sysctl defines the overcommit behaviour of the vm subsystem. The virtual memory system always does accounting of the = swap space reservation, both total for system and per-user. = Corresponding values are available through sysctl vm.swap_total, that gives the = total bytes available for swapping, and vm.swap_reserved, that gives = number of bytes that may be needed to back all currently allocated anonymous memory. Setting bit 0 of the vm.overcommit sysctl causes the virtual memory system to return failure to the process when allocation of memory = causes vm.swap_reserved to exceed vm.swap_total. Bit 1 of the sysctl = enforces RLIMIT_SWAP limit (see getrlimit(2)). Root is exempt from this = limit. Bit 2 allows to count most of the physical memory as allocatable, = except wired and free reserved pages (accounted by = vm.stats.vm.v_free_target and vm.stats.vm.v_wire_count sysctls, respectively). > #=20 > It's unclear if this implies yes or no, or even is the correct test. >=20 >>> The failures are random in the sense that restarting buildworld = either >>> produces a new assertion failure in a different library or = completion. >>>=20 >>> It isn't obvious how to capture a stack trace, if you can provide = guidance >>> I'll give it a try. As is, buildworld simply stops, the machine does = not >>> crash. >>=20 >> It might be captured for you already? I noticed files with names >> containing "symbolizer-input" and "symbolizer-ouput" like this one >>=20 >> = http://www.zefox.net/~fbsd/assertion_failure/hostname_pelorus.zefox.org/sy= mbolizer-output-7282d9 >>=20 >> and the output files contain a stack trace like this >>=20 >> llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) >> = /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:731:7 >>=20 >> llvm::sys::RunSignalHandlers() >> /usr/src/contrib/llvm-project/llvm/lib/Support/Signals.cpp:0:5 >>=20 >> SignalHandler >> /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3 >>=20 >> handle_signal >> /usr/src/lib/libthr/thread/thr_sig.c:0:3 >>=20 >> Any idea who or what is creating those files and when? >=20 > The files are deposited in /tmp, apparently by the C compiler as = records > of an internal error in the compiler, usually number 134. My = understanding=20 > is superficial at best. =20 =3D=3D=3D Mark Millard marklmi at yahoo.com