FCP 20190401-ci_policy: CI policy

[Warner Losh]

On Fri, Aug 30, 2019 at 10:25 AM Enji Cooper <yaneurabeya at gmail.com> wrote:

>
> Taking a step back, as others have brought up, we’re currently hindered by
> tooling: we are applying a DVCS (git, hg) based technique (CI) to
> subversion and testing changes after they’ve hit head, instead of before
> they hit head.
>
> While phabricator can partially solve this by testing upfront (we don’t
> enforce this; I’ve made my concerns with this not being a requirement
> well-known in the past), the solution is limited by bandwidth for testing,
> i.e., testing is an all or nothing exercise right now and building multiple
> toolchains/architectures takes a considerable amount of time. We could
> leverage cloud/distributed solutions for this (Cirrus CI, Travis if the
> integration existed), but this would require using github or teaching a
> tool how to make the appropriate REST api calls to run the tests and query
> the status (in progress, pass, fail, etc).
>
> Applying labels and filtering on test suites will get us partway to a
> final solution from a test perspective, but a lot of work needs to be done
> with phabricator, etc.
>
> We also need to have build failures with tier 1 architectures with GENERIC
> be a commit blocking operation. Full stop.
>

Until we have a DCVS, this is a non-starter. It's too hard with svn. Let's
table it until we have the migration to git complete. Also FreeBSD is huge,
having to wait for a full build on all Tier-1 platforms also is a
non-starter. It takes too long, and there's too many ways to build FreeBSD.
Having a sanity check incremental build may be OK, but there's a number of
changes that break the incremental -DNO_CLEAN build that are none-the-less
fine for a complete rebuild. There's a ton of details to get right here to
make it not an absolute nightmare for developers to get patches in and slow
the velocity of changes to a crawl. Since we know nothing of our future git
overlords, it's premature to even start this discussion because so many
things dovetail with that effort we won't get beyond the basics (which
people generally agree in principle on, but have concerns about the details
that will be filibustered to death in the absence of a concrete git system
it will add-on to).

tl;dr: Love the concept, the devil is in the details to ensure we don't
stifle momentum.

Warner