NOT_FOR_ARCHS considered harmful [was: with the cvs history? trying to help INDEX builds.]

Mark Linimon linimon at
Sat Jan 21 20:46:15 UTC 2012

On Fri, Jan 20, 2012 at 10:20:05AM +0000, Matthew Seaman wrote:
> Actually I take your point, that it should be possible to distinguish
> between ports that permanently won't work on some architectures by
> design, and ports that temporarily don't work because of mistakes or
> broken dependencies or so forth, and that are expected to be fixed
> sooner rather than later.

There's a secondary problem which I keep meaning to write up a rant
on.  This thread seems as good a place as any.

Warning: the following is only my own opinion, not portmgr's.

We made a design mistake by allowing in NOT_FOR_ARCHS as an alternative
to ONLY_FOR_ARCHS.  This has always been a shortcut to say "doesn't
build on !(amd64!i386)".  As long as the archs were (alpha|amd64|i386|
sparc64) this was merely annoying.

The problem comes when we start up package builds on new archs.   The
primary utility of package builds for tier-2s has arguably been to QA
whether the ports build at all.  The secondary utility, in order:

 - test that the arch's srcbase has't regressed to the point where it's
   too unstable to build packages

 - flag ports (and infrastructure) with bad assumptions about wordlength
   and endianness

 - create usable packages (really, only the most fundamental ports will
   have packages that are timely, due to the > 1 month cycle times)

In general, I claim if a port already has some kind of NOT_FOR_ARCH
entry, it's unlikely to build on a new arch.  This was clearly demonstrated
a couple of years ago when I first started powerpc builds once we had a
machine donated (note: powerpc = 32-bit).  The errors were correlated to
sparc64, but only roughly.

sparc64 builds tend to trip up on the following, in order of which I
think we should care:

 - 64-bit issues (which have a high correlation to amd64 build failures),

 - lack of arch-specific build stanzas (which have a high correlation to

 - endianness issues.

The powerpc build also pointed out that many ports assumed that 32bit <> i386.
On occasion, I fix notable occurences of this (e.g. python).

So, why does this matter?  Surely our sparc64 and powerpc/Mac userbases
are tiny.

The reason to do this work is that there is demand for arm and mips builds
for embedded systems[1], and once these are turned on, we're going to have
a bazillion build errors to sort through.

To the extent that our first attempts include only ports that don't have
the NOT_FOR stanzas, IMHO we're going to make more progress more quickly.

Now, for mips, only the "fundamental" ports are ever going to matter,
since there are no viable mips desktops to worry about.  But, for
embedded, getting a subset of things in net/ and sysutils/ (and to
some extent things like lang/perl) is going to be useful -- again, not
so much for uploadable packages, but to ensure buildability when vendors
try to use these in their products.

(I'm told that people are speculating about running desktop stuff on
native arm, so that's why I picked mips for the use-case.  I have no
idea if that's anything but blue-sky.)

So IMHO we should do a sweep:

 - move all the true cases of NOT_FOR_ARCHS to ONLY_FOR_ARCHS

 - move all the false cases of NOT_FOR_ARCHS to BROKEN

and then drop support for NOT_FOR_ARCHS.

If anyone is interested in coming up with patches, I'll do the -exp
run (on amd64 :-) ) to prove that there are no regressions on that
arch at least.

Finally, for those willing to investigate how messed-up the metadata
currently are, pull up the following page:

You'll find a column marked "skipped".  That contains URLs to files
called "duds.verbose" for each buildenv.  This file is the output of
'make ignorelist-verbose' called from the top of the tree.

The reason that we decided to archive these per-pointyhat-run is so
that you can tell _exactly_ why pointyhat believed that it did not have
to try to build that package at that exact point in time.  In the past,
you had to impute it from whatever the state of the tree was.  Since
the tree evolves so quickly, it was impossible to tell.  (There are
also a very small handful of degenerate cases where things don't
build on pointyhat due to its build environment.  I've worked on getting
these down to < 10 over the past few years.)

tl;dr: I want to switch the default assumption we're making.

IMHO when new ports come into the tree, we should make our default
assumption that we will try to build them on amd64 and i386.  For cases
that this does not hold, we consider this Bad and committer-must-fix.
For the tier-2s, we shift the default assumption to "only set it to
buildable once it has been shown to be so".  So, the burden of proof
shifts the other way: to a user of a tier-2 to claim "I tried this and
it works", rather than portmgr saying "we tried this and it doesn't work".

(Of course, for things like p5-* it doesn't really matter; if perl
builds, to a first approximation they'll build as well.  I'm talking
about the things like biology/, deskutils/, games/, math/, science,
x11*/, and so forth.)

What do people think?


[1] most likely in some kind of emulated instances.  Cross-builds are
left as an exercise to the reader.

More information about the freebsd-ports mailing list