NOT_FOR_ARCHS considered harmful [was: with the cvs history? trying to help INDEX builds.]

[Chris Rees]

On 21 Jan 2012 20:46, "Mark Linimon" <linimon at lonesome.com> wrote:
>
> On Fri, Jan 20, 2012 at 10:20:05AM +0000, Matthew Seaman wrote:
> > Actually I take your point, that it should be possible to distinguish
> > between ports that permanently won't work on some architectures by
> > design, and ports that temporarily don't work because of mistakes or
> > broken dependencies or so forth, and that are expected to be fixed
> > sooner rather than later.
>
> There's a secondary problem which I keep meaning to write up a rant
> on.  This thread seems as good a place as any.
>
> Warning: the following is only my own opinion, not portmgr's.
>
> We made a design mistake by allowing in NOT_FOR_ARCHS as an alternative
> to ONLY_FOR_ARCHS.  This has always been a shortcut to say "doesn't
> build on !(amd64!i386)".  As long as the archs were (alpha|amd64|i386|
> sparc64) this was merely annoying.
>
> The problem comes when we start up package builds on new archs.   The
> primary utility of package builds for tier-2s has arguably been to QA
> whether the ports build at all.  The secondary utility, in order:
>
>  - test that the arch's srcbase has't regressed to the point where it's
>   too unstable to build packages
>
>  - flag ports (and infrastructure) with bad assumptions about wordlength
>   and endianness
>
>  - create usable packages (really, only the most fundamental ports will
>   have packages that are timely, due to the > 1 month cycle times)
>
> In general, I claim if a port already has some kind of NOT_FOR_ARCH
> entry, it's unlikely to build on a new arch.  This was clearly demonstrated
> a couple of years ago when I first started powerpc builds once we had a
> machine donated (note: powerpc = 32-bit).  The errors were correlated to
> sparc64, but only roughly.
>
> sparc64 builds tend to trip up on the following, in order of which I
> think we should care:
>
>  - 64-bit issues (which have a high correlation to amd64 build failures),
>
>  - lack of arch-specific build stanzas (which have a high correlation to
>   powerpc),
>
>  - endianness issues.
>
> The powerpc build also pointed out that many ports assumed that 32bit <> i386.
> On occasion, I fix notable occurences of this (e.g. python).
>
> So, why does this matter?  Surely our sparc64 and powerpc/Mac userbases
> are tiny.
>
> The reason to do this work is that there is demand for arm and mips builds
> for embedded systems[1], and once these are turned on, we're going to have
> a bazillion build errors to sort through.
>
> To the extent that our first attempts include only ports that don't have
> the NOT_FOR stanzas, IMHO we're going to make more progress more quickly.
>
> Now, for mips, only the "fundamental" ports are ever going to matter,
> since there are no viable mips desktops to worry about.  But, for
> embedded, getting a subset of things in net/ and sysutils/ (and to
> some extent things like lang/perl) is going to be useful -- again, not
> so much for uploadable packages, but to ensure buildability when vendors
> try to use these in their products.
>
> (I'm told that people are speculating about running desktop stuff on
> native arm, so that's why I picked mips for the use-case.  I have no
> idea if that's anything but blue-sky.)
>
> So IMHO we should do a sweep:
>
>  - move all the true cases of NOT_FOR_ARCHS to ONLY_FOR_ARCHS
>
>  - move all the false cases of NOT_FOR_ARCHS to BROKEN
>
> and then drop support for NOT_FOR_ARCHS.
>
> If anyone is interested in coming up with patches, I'll do the -exp
> run (on amd64 :-) ) to prove that there are no regressions on that
> arch at least.
>
> Finally, for those willing to investigate how messed-up the metadata
> currently are, pull up the following page:
>
>  http://pointyhat.freebsd.org/errorlogs/packagestats.html
>
> You'll find a column marked "skipped".  That contains URLs to files
> called "duds.verbose" for each buildenv.  This file is the output of
> 'make ignorelist-verbose' called from the top of the tree.
>
> The reason that we decided to archive these per-pointyhat-run is so
> that you can tell _exactly_ why pointyhat believed that it did not have
> to try to build that package at that exact point in time.  In the past,
> you had to impute it from whatever the state of the tree was.  Since
> the tree evolves so quickly, it was impossible to tell.  (There are
> also a very small handful of degenerate cases where things don't
> build on pointyhat due to its build environment.  I've worked on getting
> these down to < 10 over the past few years.)
>
> tl;dr: I want to switch the default assumption we're making.
>
> IMHO when new ports come into the tree, we should make our default
> assumption that we will try to build them on amd64 and i386.  For cases
> that this does not hold, we consider this Bad and committer-must-fix.
> For the tier-2s, we shift the default assumption to "only set it to
> buildable once it has been shown to be so".  So, the burden of proof
> shifts the other way: to a user of a tier-2 to claim "I tried this and
> it works", rather than portmgr saying "we tried this and it doesn't work".
>
> (Of course, for things like p5-* it doesn't really matter; if perl
> builds, to a first approximation they'll build as well.  I'm talking
> about the things like biology/, deskutils/, games/, math/, science,
> x11*/, and so forth.)
>
> What do people think?
>

I think we'll end up with a FreeBSD that only works on amd64 and IA-32...

Chris