Schedule for releases

Garrett Cooper gcooper at
Wed Dec 29 11:17:43 UTC 2010

On Tue, Dec 28, 2010 at 3:01 PM, Alexander Leidinger
<Alexander at> wrote:


>> I am still not convinced that whatever development model people and
>> companies use (and I heard of in here) is better than to just devel
>> on HEAD and if it works there merge it and backport it to your release
>> branch for QA and shipping.
> It may not be a problem for developers which know enough about FreeBSD,
> but try to sell this to people which do not know enough about FreeBSD
> or some management-people (and I'm not talking about the
> money-argument here).

Bottom line is always feature set in products and ship date from what
I've seen, unless someone understands the implications of licensing
and takes a look at the big picture as far as maintainability is
concerned. Or maybe that little culture is just Cisco and their desire
for filling in pretty checkboxes based on what marketing thinks that
users want driven by a program management team and managers *shrug*.

>From what I see many people like Linux because it's an established
name in free software, there are support companies that `reduce' the
need to maintain software because they produce the packages, tools,
and infrastructure to make your life `easier' (Android SDK,
Montavista, etc). and there are N times many code monkeys in the world
that `know' how do hack a Linux distro fifteen different ways to
Sunday, compared to developers who work within the opensource
communities. Because people see a prototype out the door, and can
assemble projects faster with OS X.Y.Z, they usually go with that over
something else (sometimes BSD, sometimes not).

>> We still lack the parts that would tell us something in the last week
>> or last 24 hours caused a regression that made my TCP/NFS/ZFS/UFS/<you
>> name it> n% slower.  Kris had been doing a good job in the past but as
>> time shows we need more people, different setups, ...
> We do not lack the parts, we lack someone to take the parts and get
> them up and running. See:

No: according to Erik in the first link, the parts were never fully
integrated and the project as a whole was never completed due to lack
of hardware and time. Some bits are also just directions as well as
Erik says in his reply to the thread. After seeing enough folks
prematurely claim victory within my own company, I know enough that
nothing is complete unless it works (to some basic degree, i.e. it can
be replicated by someone other than the original author of the
infrastructure or code using the directions provided from scratch).

>> It's not only "compiles", "boots", but also the formerly in this
>> thread mentioned "works correctly" and in addition to that the "works
>> well as expected" or "works better than before" - hopefully;).
> Maybe this could also be used to run the regression tests as one of the
> benchmarks. If yes: As Robert mentioned, we can not go and tell to run
> them all in one command (ATM), but we could have each of them as a
> different benchmark.

Regression tests should be stable. Regression tests shouldn't crash
targets (in general). Benchmarks and stress tests can do that. We need
regression tests that can be run on a nightly/weekly/monthly/whatever
basis with the needed level of breadth and depth on a dedicated set of
cluster machines that we can gather results with in a sane manner
otherwise whoever is managing the machines will run into the
maintenance nightmare that portmgr was/is running into with the
tinderbox machines on a regular/periodic basis (ask Mark Linimon about
that). ports infrastructure is partly to blame, yes, but a large part
of things as well is infrastructure tied up in pointyhat, etc as far
as was explained / conveyed to me over email and IRC. As much as I
love Yahoo! and the infrastructure work that Yahoo! has provided, a
lot of what I see wrong with how things are done with some of the work
that Kris did is that it was never made portable, and documentation is
lacking in areas -- thus duplicating infrastructure, results, success,
and profit is difficult.

IMO we should have something like:

make checkworld
make checkkernel

that would go off and run functional regression tests, and something like:

make stressworld
make stresskernel

that would to similar for benchmarks as well. Having stuff tied
together between the sources and somewhere in the tree is a minor
infrastructure hurdle to get over, but a must for integration. kyua
(because ATF has been abandoned as a project except for maintenance
releases and bugfixes are concerned) should be integrated in base as
well as a tool. Giorgios has engaged Julio from NetBSD about it in the
past couple of weeks and I will follow up again as well, time
permitting. Adding it to ports is quite frankly a waste of time. That
way once the infrastructure is in place, we can easily port over
testcases from NetBSD (technically we could do the same right now
using the ATF testcases and that might be a good short-term investment
even though it's sort of a wasted effort in one respect), and get the
coverage that we need on a regular basis to increase confidence in
FreeBSD as far as stability on HEAD and MFCs is concerned, so
developers can instead focus on writing more code and tests
corresponding to that code, so FreeBSD can better scale longterm as a


Warning: the above is based on personal opinion as IANAPGM (not a
program manager :P), thank my lucky stars...

More information about the freebsd-arch mailing list