Quality of FreeBSD

Mark Linimon linimon at lonesome.com
Thu Jul 21 21:15:32 GMT 2005

On Thu, Jul 21, 2005 at 08:00:40PM +0100, Robert Watson wrote:
> [original poster wrote:]
> >- I completely agree with MikeM - any kind of complex software could be
> >tested with right prepared test cases, specially if they are going to be
> >reused in the next release;

For static problems -- yes.  For dynamic problems, such as race conditions,
the problem space you are trying to test is many orders of magnitude more
complex.  This is true of any engineering discipline, but much more so
with software engineering due to the immense complexity of the constructed

> [rwatson again:]
> As has been discussed extensively in this thread and other threads, the
> FreeBSD development model typically addresses change at the tree HEAD,
> where the changes are tested and evaluated, and then they are back-ported.
> Some changes are low-risk, and are backported quickly (minor locking
> fixes, error handling, etc).  Others are higher risk, and are backported
> only when they are felt to have received sufficient testing (driver
> re-writes, structural changes).  Other changes are considered too large
> to ever be backported [ ... ]

To add to Robert's comments, there was at least one case during the 5.2
cycle where a large backport was made that destabilized the tree for quite
some time.  This was not due to any lack of diligence on the developer's
part; it turned out that the problems were far more subtle and complex
than anyone could have reasonably anticipated.  Since that time, AFAICT
the sentiment has shifted away from large backports.

There is always risk in any backport and the risks escalate dramatically
the less compartmentalized the changes are.  One of the goals for 6.X
and beyond is to try to keep changes more compartmentalized; there was
simply no way to do such a thing with e.g. SMP and VM changes.  At the
same time, the sentiment seems to be "let's debug one set of featureset
changes all together and then release them as a major release".

Of course, backports also require developer time both to do the initial
commit and then, more onerously, the followup support.

To conclude this thought, the motivation for changing the way FreeBSD
is going to do releases going forwards is to try to mitigate such
problems: to try to debug, and release, a smaller set of features with
new major releases, and more frequently, and with a better-known
schedule (every 18 months).

> Notice that we'll be supporting the 4.x branch for several years to come.

The limiting factor on the 4.X branch is going to be the ports tree
more quickly than the base system, particularly for people running
desktop installations.  The FreeBSD GNOME team has already announced
that they are not going to support 4.X by default in the next major
GNOME release due this fall.  The next major KDE release will probably
not work on the 4.X gcc compiler as well IIUC.  There are simply an
insufficient amount of developer resources to support releases that
have different toolchains, include files, and so on.

Staying on 4.X indefinitely is not going to be an option at some point
in the future, but when, exactly, is difficult to tell right now.  It
is fair to note, however, that almost no developer attention is being
spent on 4.X except for security problems as they are found.

Further, the more people we have stay on 4.X, the less people we have
testing whichever release we consider the latest "stable" release, and
therefore, the less bugs we'll get fixed on that release.

One last thought.  It always bears repeating that, except for a handful
of cases, people who work on FreeBSD are not being paid to do so.  Users
should always adjust their expectations accordingly.  We do our absolute
best given the relatively small number of developers that we do have,
but we always need more people who are willing to work on regression
testing and QA activities.  For the companies which view FreeBSD as
'mission-critical' (and we do welcome them!), I challenge them to
consider funding development/testing efforts going forwards.  (Yes, a
number already do, but more would be welcome.)


More information about the freebsd-stable mailing list