From nobody Sat Sep 13 16:13:19 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cPGYn0cmgz66gfw for ; Sat, 13 Sep 2025 16:13:33 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cPGYm4sXWz3qKS for ; Sat, 13 Sep 2025 16:13:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-329b760080fso2932166a91.1 for ; Sat, 13 Sep 2025 09:13:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1757780010; x=1758384810; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=L1V6B528BjXCAmu5i4uF1Hdd0E3ixTuzlN/YmNgPWd0=; b=NLbxxmxoSWutBJveC4WtMafZnuavjDxKE+8DLtR68CDvrZ4eWRRi+j88aUd/K1TxgO 6vJM8lAQMSJFCGGn469ZWsgJsZpuJfSmzO8I20HlvFxzPePF+JU/sY/aO+xcjENBRpWP xbFHj69XFBcS/kXL4Zf1V5rqfSI9PuQP1UAOLQv0OcAiB2o1xNfV0xN710gsvg6SJXlD vskdg2lqaTbWJqk8oC5bH4l0yayDRPv9C+Yzo4GZNC680ektZK71AhHrkiH+RwMDyvRh 97dyfQAejkrlw/iWiNbhbPga+if5XQI5XqzJcTFLlOPLVw1ObmwUi0qqMyca+4eF9NEy FihQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757780010; x=1758384810; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=L1V6B528BjXCAmu5i4uF1Hdd0E3ixTuzlN/YmNgPWd0=; b=NXlXBxariu8R98p1xSHkO7Wb9I4hq0zNBj9lI1WA3bW5Rv/84aAtqiiJg+9Vnf+OO9 drE4s8JMr004F4RScFso2aso63X2jE3HRCY7Gab1lOOtVaEucL/mLAmv9TRMH43/UvWN 1VO7Ev3TlHFNI2pnipemhL6DntfSVyJoENOCO/aGo2Svbm1QqvWK+gNZBY+k2ihWjvnQ 4eiCZT4FFUohyMP/3C/h4V5R+/Jh2nBa7PNlvY7de9ZTuHFfQ/V+AWvx1Pep9XKUIKF2 b0V42Md9xpNs1Wn8gdNTjYTsrmm1Pe8aLKiq10ThxtWhzvyOKMAojWNjjQgwW+fWE0xV 2Opg== X-Forwarded-Encrypted: i=1; AJvYcCXnUv6yPX9TvVGrKloDKW44JB0tjkBRs3HMEZKewGA9HCLcI2iOBKDgkdBGxi3FD3P/TprTwlOlqOtEiGY0BmM=@freebsd.org X-Gm-Message-State: AOJu0Yxq35v61LH/uF047jEe6X6XW5BHmKrU7hqjXFSfxQaLtpVvjll1 F3pAISB9KJ2iUgDfGFjIYgJlNXIZ52Pr7ChE1ePTLpzunBgve/7Dfhk/gjQWUZzDdMAv7Nk4mSD k8Ikyh1qegrTCvyss/yHu/do6bO2CSuiJ4bokLsA3Mg== X-Gm-Gg: ASbGnctpxukthhNKLu1+U3qKQqxavwZZ/4w9Sw/i3L405kSj4rVGEGyLc+Iv9KtMiUk K1NwHudjartRoE+0LMYokfmoP09acYoXA7WUbYosE6azaMP+5/yJbUqeya2J07yv/YRbRyIAjfA mLkQ/E7r0sIBjYGuzOQEzscXrbOjo8lgGw50wbTdksU6wwQ1Ey5iXiMZ1E2HdUzonW4h8f+vgDN vz8gVhsV1RnCL4M+1dGQslU4Hn8e8GyJ7rEZPg= X-Google-Smtp-Source: AGHT+IFeNk1j+NA1oy3uJhZcO0H2JGmrWQ9jHiL5Y5i5Oo51E986irYdhM2JEqJ7fCuLbzTBRiJIFuBDii8DTM8Ax+o= X-Received: by 2002:a17:90b:2f4d:b0:32b:d183:facf with SMTP id 98e67ed59e1d1-32de4f98573mr7488765a91.28.1757780010423; Sat, 13 Sep 2025 09:13:30 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sat, 13 Sep 2025 10:13:19 -0600 X-Gm-Features: Ac12FXzui3248Tsgiz9GfNcJ90hlLeCC7F0BF-W8QIaPPc075z9seNfAzCa4wM4 Message-ID: Subject: Re: Git and buildworld running at the same time To: bob prohaska Cc: Brad Davis , FreeBSD Current Content-Type: multipart/alternative; boundary="0000000000001d6c0a063eb10d83" X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4cPGYm4sXWz3qKS --0000000000001d6c0a063eb10d83 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Sep 13, 2025 at 10:08=E2=80=AFAM bob prohaska = wrote: > On Sat, Sep 13, 2025 at 09:53:46AM -0600, Brad Davis wrote: > > On Sat, Sep 13, 2025, at 9:40 AM, Warner Losh wrote: > > > > > > > > > On Sat, Sep 13, 2025 at 9:28=E2=80=AFAM Brad Davis = wrote: > > >> On Sat, Sep 13, 2025, at 8:57 AM, bob prohaska wrote: > > >> > Lately I've noticed that sometimes while running buildworld a top > > >> > window reports git running also. Up to now, I've surmised that > > >> > this is intentional, with git providing some housekeeping function= . > > >> > > > >> > Yesterday a buildworld session was accompanied by a prolonged > > >> > interval of git running also, with a large memory footprint, > > >> > near 1GB. That seems rather excessive. > > >> > > > >> > At the same time, it dawned on me that my recent habit has been > > >> > to run git pull, immediately followed by buildworld. Might it be > > >> > prudent to wait (how long?) to let git finish any housekeeping > > >> > triggered by the pull command? It seems likely that any overlap > > >> > could readily lead to inconsistencies which might account for > > >> > some of the buildworld problems I've been encountering lately. > > >> > > >> This is part of the normal build process and how the output of uname > -a includes bits like this: main-n280188-2024887abc7d-dirty or > main-n280188-2024887abc7d > > >> > > >> To find out of the src tree is pristine or dirty the build process > uses git to find out. > > > > > > Though that's only for the kernel, not for world builds. Right? > > > > It happens for world builds as well.. newvers.sh runs git (used to be > svnlite) and is called in Makefile.inc1. > > > > Is it normal for git to consume (while buildworld is running) close > to 1 GB of memory for durations exceeding an hour? > > More fundamentally, is it OK to start buildworld immediately after > a git pull finishes interactive output and returns a shell prompt? > It's what I do: % git pull --rebase % make buildworld I don't think it's normal for hours-long git runs, even with just 1GB. That's why I'm so interested in this. Why is git even running during buildworld? It shouldn't. That's a bug in my mind. I sometimes have git take minutes to do garbage collection on a rebase. But that's inline, not in the background. So some more basic questions to help me understand: (1) what root are you running on (media type and filesystem) (2) can you capture the full git command that's taking hours? (3) Any extra, non-default settings? (4) What's the exact build command you are using? Warner --0000000000001d6c0a063eb10d83 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Sep 13,= 2025 at 10:08=E2=80=AFAM bob prohaska <fbsd@www.zefox.net> wrote:
On Sat, Sep 13, 2025 at 09:53:46AM -0600, Brad Davi= s wrote:
> On Sat, Sep 13, 2025, at 9:40 AM, Warner Losh wrote:
> >
> >
> > On Sat, Sep 13, 2025 at 9:28=E2=80=AFAM Brad Davis <brd@freebsd.org> wrote: > >> On Sat, Sep 13, 2025, at 8:57 AM, bob prohaska wrote:
> >> > Lately I've noticed that sometimes while running bui= ldworld a top
> >> > window reports git running also. Up to now, I've sur= mised that
> >> > this is intentional, with git providing some housekeepin= g function.
> >> >
> >> > Yesterday a buildworld session was accompanied by a prol= onged
> >> > interval of git running also, with a large memory footpr= int,
> >> > near 1GB. That seems rather excessive.
> >> >
> >> > At the same time, it dawned on me that my recent habit h= as been
> >> > to run git pull, immediately followed by buildworld. Mig= ht it be
> >> > prudent to wait (how long?) to let git finish any housek= eeping
> >> > triggered by the pull command? It seems likely that any = overlap
> >> > could readily lead to inconsistencies which might accoun= t for
> >> > some of the buildworld problems I've been encounteri= ng lately.
> >>
> >> This is part of the normal build process and how the output o= f uname -a includes bits like this: main-n280188-2024887abc7d-dirty or main= -n280188-2024887abc7d
> >>
> >> To find out of the src tree is pristine or dirty the build pr= ocess uses git to find out.
> >
> > Though that's only for the kernel, not for world builds. Righ= t?
>
> It happens for world builds as well.. newvers.sh runs git (used to be = svnlite) and is called in Makefile.inc1.
>

Is it normal for git to consume (while buildworld is running) close
to 1 GB of memory for durations exceeding an hour?

More fundamentally, is it OK to start buildworld immediately after
a git pull finishes interactive output and returns a shell prompt?

It's what I do:

% = git pull --rebase
% make buildworld <args>

I don't think it's normal for hours-long git runs, even wi= th just 1GB. That's why I'm so interested in this. Why is git even = running during buildworld? It shouldn't. That's a bug in my mind.

I sometimes have git take minutes to do garbage col= lection on a rebase. But that's inline, not in the background.

So some more basic questions to help me understand:
<= div>(1) what root are you running on (media type and filesystem)
= (2) can you capture the full git command that's taking hours?
(3) Any extra, non-default settings?
(4) What's the exact bu= ild command you are using?

Warner
--0000000000001d6c0a063eb10d83--