Re: arm64 pkg building is taking longer

Mark, good morning!

The bind920 package, in the quarterly branch, is still unavailable for
ARM64. We already have a new version of bind920 committed 9.20.6.
Mat committed the Security fixes on Feb 3rd. I already fixed it by building
it myself, but I am thinking about the broad community and the increase of
the adoption of ARM64 architecture.

Considering the challenges that you described, do you think more ampere
servers would help? If so, maybe we should work with The FreeBSD
Foundations to try to get these resources for this project as part of some
sponsorship.

Thanks,

Diego Linke


On Mon, Feb 10, 2025 at 7:22 PM Diego Linke <diego@bsd.com.br> wrote:

> Hi Mark,
>
> Thank you very much for the detailed explanation.
>
> I appreciate the insights into the differences in build resources and the
> constraints affecting ARM64. I also appreciate that security updates are
> already prioritized within the available resources.
>
> Regards,
>
> Diego Linke
>
>
> On Mon, Feb 10, 2025 at 6:48 PM Mark Millard <marklmi@yahoo.com> wrote:
>
>> Diego Linke <diego_at_bsd.com.br> wrote on
>> Date: Mon, 10 Feb 2025 12:07:19 UTC :
>>
>> > I noticed that ARM64 packages take a few days longer to build and become
>> > available on the official FreeBSD servers compared to AMD64 and i386.
>>
>> The amd64 systems are generally faster and there are
>> a lot more of them used as package builders.
>>
>> There are only 3 aarch64 build systems: ampere[1-3].
>>
>> By contrast, there are 10 amd64 build systems, 3 being
>> fairly modern and vastly faster than the ampere*'s
>> (far more hardware threads, for example).
>>
>> ampere1 cycles through building and distributing:
>> 141arm64-quarterly
>> 141releng-armv7-quarterly
>> 1341arm64-quarterly
>> 134releng-armv7-quarterly
>>
>> amd64 systems do not build that many variations on the
>> same machine: in fact each normally only builds one
>> variation, no waiting for other cycle members to
>> finish.
>>
>> As each takes longer, the next time it gets back to the
>> same type of build, it is likely that far more packages
>> that need to be built (compared to the analogous amd64
>> context). It is not as extreme for quarterly as it is
>> for default (a.k.a. latest).
>>
>> ampere3 is similar (default here is a.k.a. latest):
>> 141arm64-default
>> 141releng-armv7-default
>> 1341arm64-default
>> 134releng-armv7-default
>>
>> ampere3 likely builds the most per poudriere bulk run
>> compared to ampere1 above, taking the largest to get
>> back to the next build of the same member of the cycle.
>>
>> ampere2 has only 2 cycle members (as stands, main is 15.0):
>> main-arm64-default
>> main-armv7-default
>>
>> So that makes 10 separate variations overall, spread
>> over the 3 ampere*'s.
>>
>> amd64/i386 has 10 separate variations as well, but
>> 1 per amd64 system that does the type build.
>> 141amd64-quarterly , 141amd64-default , and
>> 141i386-default are what get the fastest 3 builder
>> machines.
>>
>> > For example, last Monday, Mat committed the Bind 9.2 (dns/bind920)
>> update
>> > to 9.20.5 in the quarterly branch, which has two security fixes. This
>> > update was available on amd64 and i386 2 days later. It's still not
>> > available (after 7 days) for ARM64.
>>
>> Expected sort of result, given the resources available to put
>> to use.
>>
>> > Could we prioritize building packages with security updates, especially
>> for
>> > the quarterly branch?
>>
>> Already done: The more and bigger "default" builds do not
>> complete for the machine time on ampere1. Mixed on the
>> same machine the "default" builds would further delay the
>> quarterly builds.
>>
>> > Is anyone aware of any initiatives to improve this
>> > process?
>>
>> Unless the aarch64/armv7 system resources are considered
>> as part of the "process": it is not basically a process
>> problem. (I'm not intending to imply that "no optimization
>> is possible", just that such is not likely to lead to
>> a large change of scale for how long things take in
>> order to reach similar times to amd64 now takes.)
>>
>> > PS: I’m aware that I can set up my own package build infrastructure
>> using
>> > Poudriere and am considering it as a fallback option. However, I’d like
>> to
>> > explore whether we can improve the process for everyone.
>>
>> That last note repeats here: it is not basically a process
>> problem for what is the major constraint.
>>
>>
>>
>> ===
>> Mark Millard
>> marklmi at yahoo.com
>>
>>