Re: Git and buildworld running at the same time

On Sat, Sep 13, 2025 at 10:13:19AM -0600, Warner Losh wrote:
> On Sat, Sep 13, 2025 at 10:08 AM bob prohaska <fbsd@www.zefox.net> wrote:
> 
> > On Sat, Sep 13, 2025 at 09:53:46AM -0600, Brad Davis wrote:
> > > On Sat, Sep 13, 2025, at 9:40 AM, Warner Losh wrote:
> > > >
> > > >
> > > > On Sat, Sep 13, 2025 at 9:28 AM Brad Davis <brd@freebsd.org> wrote:
> > > >> On Sat, Sep 13, 2025, at 8:57 AM, bob prohaska wrote:
> > > >> > Lately I've noticed that sometimes while running buildworld a top
> > > >> > window reports git running also. Up to now, I've surmised that
> > > >> > this is intentional, with git providing some housekeeping function.
> > > >> >
> > > >> > Yesterday a buildworld session was accompanied by a prolonged
> > > >> > interval of git running also, with a large memory footprint,
> > > >> > near 1GB. That seems rather excessive.
> > > >> >
> > > >> > At the same time, it dawned on me that my recent habit has been
> > > >> > to run git pull, immediately followed by buildworld. Might it be
> > > >> > prudent to wait (how long?) to let git finish any housekeeping
> > > >> > triggered by the pull command? It seems likely that any overlap
> > > >> > could readily lead to inconsistencies which might account for
> > > >> > some of the buildworld problems I've been encountering lately.
> > > >>
> > > >> This is part of the normal build process and how the output of uname
> > -a includes bits like this: main-n280188-2024887abc7d-dirty or
> > main-n280188-2024887abc7d
> > > >>
> > > >> To find out of the src tree is pristine or dirty the build process
> > uses git to find out.
> > > >
> > > > Though that's only for the kernel, not for world builds. Right?
> > >
> > > It happens for world builds as well.. newvers.sh runs git (used to be
> > svnlite) and is called in Makefile.inc1.
> > >
> >
> > Is it normal for git to consume (while buildworld is running) close
> > to 1 GB of memory for durations exceeding an hour?
> >
> > More fundamentally, is it OK to start buildworld immediately after
> > a git pull finishes interactive output and returns a shell prompt?
> >
> 
> It's what I do:
> 
> % git pull --rebase
> % make buildworld <args>
> 
> I don't think it's normal for hours-long git runs, even with just 1GB.
> That's why I'm so interested in this. Why is git even running during
> buildworld? It shouldn't. That's a bug in my mind.
> 
> I sometimes have git take minutes to do garbage collection on a rebase. But
> that's inline, not in the background.
> 
> So some more basic questions to help me understand:
> (1) what root are you running on (media type and filesystem)
The machines are Raspberry Pi 2, so ARMv7, using a USB hard disk
with UFS filesystems.

> (2) can you capture the full git command that's taking hours?
The git command I normally invoke manully has been a plain
git pull
with no added arguments. Once it returns to a shell prompt I start
make -j3   KERNCONF=GENERIC  TARGET=arm TARGET_ARCH=armv7 buildworld > buildworld.log && make -j4  KERNCONF=GENERIC buildkernel  > buildkernel.log && etcupdate -p > etcupdate.log
and put it in the background.

I've been noticing the git command running in top sessions, but have not
been able to make out any details of the git command, for example gc --auto.
I'll check using ps next time git makes an untimely appearance.
 

> (3) Any extra, non-default settings?
root@www:/usr/src # more /etc/src-env.conf
WITH_META_MODE=YES


> (4) What's the exact build command you are using?
As above, though I tinker with -j settings for make buildworld.
Usually it's -j3, but sometimes -j4 works. Buildworld is prone
to silently hanging, forcing a power-cycle and repeat which usually
runs to completion. If not, I try -j2.
 

Thanks for writing!
bob prohaska