From nobody Sat Sep 13 16:34:08 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cPH1X2FH0z66j45 for ; Sat, 13 Sep 2025 16:34:08 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pelorus.zefox.org", Issuer "pelorus.zefox.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cPH1W6fx6z3tnB; Sat, 13 Sep 2025 16:34:07 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Authentication-Results: mx1.freebsd.org; none Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.18.1/8.18.1) with ESMTPS id 58DGY9TW021433 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 13 Sep 2025 09:34:09 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.18.1/8.18.1/Submit) id 58DGY9kn021432; Sat, 13 Sep 2025 09:34:09 -0700 (PDT) (envelope-from fbsd) Date: Sat, 13 Sep 2025 09:34:08 -0700 From: bob prohaska To: Warner Losh Cc: Brad Davis , FreeBSD Current , bob prohaska Subject: Re: Git and buildworld running at the same time Message-ID: References: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4cPH1W6fx6z3tnB On Sat, Sep 13, 2025 at 10:13:19AM -0600, Warner Losh wrote: > On Sat, Sep 13, 2025 at 10:08 AM bob prohaska wrote: > > > On Sat, Sep 13, 2025 at 09:53:46AM -0600, Brad Davis wrote: > > > On Sat, Sep 13, 2025, at 9:40 AM, Warner Losh wrote: > > > > > > > > > > > > On Sat, Sep 13, 2025 at 9:28 AM Brad Davis wrote: > > > >> On Sat, Sep 13, 2025, at 8:57 AM, bob prohaska wrote: > > > >> > Lately I've noticed that sometimes while running buildworld a top > > > >> > window reports git running also. Up to now, I've surmised that > > > >> > this is intentional, with git providing some housekeeping function. > > > >> > > > > >> > Yesterday a buildworld session was accompanied by a prolonged > > > >> > interval of git running also, with a large memory footprint, > > > >> > near 1GB. That seems rather excessive. > > > >> > > > > >> > At the same time, it dawned on me that my recent habit has been > > > >> > to run git pull, immediately followed by buildworld. Might it be > > > >> > prudent to wait (how long?) to let git finish any housekeeping > > > >> > triggered by the pull command? It seems likely that any overlap > > > >> > could readily lead to inconsistencies which might account for > > > >> > some of the buildworld problems I've been encountering lately. > > > >> > > > >> This is part of the normal build process and how the output of uname > > -a includes bits like this: main-n280188-2024887abc7d-dirty or > > main-n280188-2024887abc7d > > > >> > > > >> To find out of the src tree is pristine or dirty the build process > > uses git to find out. > > > > > > > > Though that's only for the kernel, not for world builds. Right? > > > > > > It happens for world builds as well.. newvers.sh runs git (used to be > > svnlite) and is called in Makefile.inc1. > > > > > > > Is it normal for git to consume (while buildworld is running) close > > to 1 GB of memory for durations exceeding an hour? > > > > More fundamentally, is it OK to start buildworld immediately after > > a git pull finishes interactive output and returns a shell prompt? > > > > It's what I do: > > % git pull --rebase > % make buildworld > > I don't think it's normal for hours-long git runs, even with just 1GB. > That's why I'm so interested in this. Why is git even running during > buildworld? It shouldn't. That's a bug in my mind. > > I sometimes have git take minutes to do garbage collection on a rebase. But > that's inline, not in the background. > > So some more basic questions to help me understand: > (1) what root are you running on (media type and filesystem) The machines are Raspberry Pi 2, so ARMv7, using a USB hard disk with UFS filesystems. > (2) can you capture the full git command that's taking hours? The git command I normally invoke manully has been a plain git pull with no added arguments. Once it returns to a shell prompt I start make -j3 KERNCONF=GENERIC TARGET=arm TARGET_ARCH=armv7 buildworld > buildworld.log && make -j4 KERNCONF=GENERIC buildkernel > buildkernel.log && etcupdate -p > etcupdate.log and put it in the background. I've been noticing the git command running in top sessions, but have not been able to make out any details of the git command, for example gc --auto. I'll check using ps next time git makes an untimely appearance. > (3) Any extra, non-default settings? root@www:/usr/src # more /etc/src-env.conf WITH_META_MODE=YES > (4) What's the exact build command you are using? As above, though I tinker with -j settings for make buildworld. Usually it's -j3, but sometimes -j4 works. Buildworld is prone to silently hanging, forcing a power-cycle and repeat which usually runs to completion. If not, I try -j2. Thanks for writing! bob prohaska