From nobody Fri Nov 14 15:25:27 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4d7LXP2qVrz6GbX5; Fri, 14 Nov 2025 15:24:21 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pelorus.zefox.org", Issuer "pelorus.zefox.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4d7LXN6VWWz3csn; Fri, 14 Nov 2025 15:24:20 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Authentication-Results: mx1.freebsd.org; none Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.18.1/8.18.1) with ESMTPS id 5AEFPRXt038077 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 14 Nov 2025 07:25:28 -0800 (PST) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.18.1/8.18.1/Submit) id 5AEFPR40038076; Fri, 14 Nov 2025 07:25:27 -0800 (PST) (envelope-from fbsd) Date: Fri, 14 Nov 2025 07:25:27 -0800 From: bob prohaska To: Carl Shapiro Cc: Ronald Klop , freebsd-arm@freebsd.org, freebsd-current@freebsd.org Subject: Re: Still seeing Failed assertion: "p[i] == 0" on armv7 buildworld Message-ID: References: <4957be52-e57f-4f5f-9626-d0f706480fe1@FreeBSD.org> <87ldkai9lu.fsf@panix.com> <877bvthymv.fsf@panix.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <877bvthymv.fsf@panix.com> X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4d7LXN6VWWz3csn On Thu, Nov 13, 2025 at 11:16:56PM -0800, Carl Shapiro wrote: > bob prohaska writes: > > > All the assertion failures I've seen have been in the clang libraries during > > buildworld. They appear to happen in a variety of cases, indicated by the > > different .sh and .cpp filenames found in the files under > > http://www.zefox.net/~fbsd/assertion_failure/ > > Do you have the stdout and stderr of the build somewhere in there as > well? The make(1) invocation in the readme file shows its output being > redirected to a file. Those files have been overwritten by restarting the buildworld sessions. They tend to be large and diffcult to synchronize with the .cpp and .sh files generated by the crash. It could be done if it's useful. > > The assert you mentioned in the subject of your e-mail message, which I > also saw in the readme file, could come from jemalloc. See these lines > of code for the context > > https://github.com/facebook/jemalloc/blob/dev/src/extent.c#L805-L814 > > That assertion will be tripped when jemalloc sees non-zero memory that > it expects to be zeroed. See for example > > https://github.com/facebook/jemalloc/blob/dev/src/pages.c#L55-L106 > > Looking at the code, my hypothesis would be that jemalloc thinks it's > committing memory for the first time but the memory is coming back with > non-zero data. > > Just curious, but is over-commit enabled on your system? Here is the > signal jemalloc is using to check > > https://github.com/facebook/jemalloc/blob/dev/src/pages.c#L729-L737 > Sysctl -a reports in part: # sysctl -a | grep -i overcommit sysctl: S_vmtotal 48 != 88 vm.overcommit: 0 # It's unclear if this implies yes or no, or even is the correct test. > > The failures are random in the sense that restarting buildworld either > > produces a new assertion failure in a different library or completion. > > > > It isn't obvious how to capture a stack trace, if you can provide guidance > > I'll give it a try. As is, buildworld simply stops, the machine does not > > crash. > > It might be captured for you already? I noticed files with names > containing "symbolizer-input" and "symbolizer-ouput" like this one > > http://www.zefox.net/~fbsd/assertion_failure/hostname_pelorus.zefox.org/symbolizer-output-7282d9 > > and the output files contain a stack trace like this > > llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) > /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:731:7 > > llvm::sys::RunSignalHandlers() > /usr/src/contrib/llvm-project/llvm/lib/Support/Signals.cpp:0:5 > > SignalHandler > /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3 > > handle_signal > /usr/src/lib/libthr/thread/thr_sig.c:0:3 > > Any idea who or what is creating those files and when? The files are deposited in /tmp, apparently by the C compiler as records of an internal error in the compiler, usually number 134. My understanding is superficial at best. Thanks for writing! bob prohaska