Re: FYI: pkg 2.2.1 and official main-amd64 package builder beefy18's time frames: good news and bad news (preliminary)

On Jul 6, 2025, at 13:48, Mark Millard <marklmi@yahoo.com> wrote:

> [Sending to the right list this time, not to the one
> for PkgBase.]
> 
> On Jul 6, 2025, at 13:29, Mark Millard <marklmi@yahoo.com> wrote:
> 
> Context for these comments:
> 
> https://pkg-status.freebsd.org/beefy18/build.html?mastername=main-amd64-default&build=pe02807e84be6_s24ebe9fa0ec
> 
> as of having built 31634 packages (plus 122 failed),
> 4336 or so packages Remaining to attempt to finish.
> 
> I'll note that the example being main-amd64, it
> involves the debug FreeBSD kernel. That mixed with
> the pkg 2.1+ or pkg 2.2+ involvement make for things
> taking a lot longer to build overall.
> 
> 
> The good news: The built-pkg-count vs. time curve
> structure generally looks much closer to linear for
> the package count involved so far, unlike the concave
> up structure that pkg 2.1.* lead to. Concave-up cannot
> be viable long term as the number of packages
> to build increases. The scale for that issue has
> shifted signifcantly in the better direction.
> 
> (There will be better data about this once the overall
> build completes.)
> 
> 
> The bad news: The basically linear rate for 2.2.1
> (so far) vs. 2.0.6 for the fairly modern pkgs built
> counts ends up with:
> 
> Official build examples on beefy18:
> Newer, pkg 2.2.1 involvement: Built 31634 (so far) in elapsed-time 210.3 hrs or so.
> Older, pkg 2.0.6 involvement: Built 35771          in elapsed-time  96.5 hrs or so.
> 
> 
> So: the overall build had already taken more than
> twice as long as it used to and still had 4336
> to attempt to finish building.
> 
> 
> ampere2'main-arm64 and main-armv7 take noticeably
> longer on the slower hardware. So the above for
> main-amd64 (beefy18) is likely not the worst case.
> 
> But, as ampere2 is still working on finishing with
> pkg 2.1.4 for main-armv7 builds before it starts
> building main-arm64 with pkg 2.2.1 involved, it
> might be 4+ weeks to have that 2.2.1 example
> complete and main-armv7 with pkg 2.2.1 involved
> start. As stands:
> 
> Built 16638 Remaining 16109 but Elapsed 154:43:03.
> (So it suggests another 150 or so hrs overall as a
> quick estimate for it finishing the pkg 2.1.4 based
> example.)
> 
> 
> Note:
> Builds of main-i386 (beefy17) are scheduled to stop.
> So I've ignored it here, although it looks similar
> to the figures for main-amd64 (beefy18).

As things have progressed, it is now clear that
2.2.1 still has a concave up structure for this
debug-kernel (main-amd64) context, just with
less curvature compared to the various 2.1.*
versions, the actual comparison involved being
against 2.1.2. (Faster systems make this harder
to see. Slower ones make it easier to see.)

As stands, at 33558 built, it has already taken
somewhat over 2.3 times longer to get that far
than using 2.0.6 took to finish, having built
35958 packages in the process with 2.0.6 involved.


For ampere2's main-arm64/main-armv7 cycle, I'd not
be surprised if each of those takes around 2 weeks
when it is a (nearly) full rebuild of the packages.
So a full cycle of such might take about a month.
The longer intervals between builds of a specific
type makes large rebuilds more common.

It is too bad there could not be 2 builder machines
for the main-arm64/main-armv7 pair: one using the
non-debug kernel for more frequent distribution
builds and one using the debug-kernel for testing
main.

===
Mark Millard
marklmi at yahoo.com