Quality of FreeBSD

[Garance A Drosihn]

At 8:50 AM -0400 7/21/05, MikeM wrote:
>On 7/21/2005 at 8:29 PM Daniel O'Connor wrote:
>|
>| I think the best way to rectify this is to test RC candidates
>| on YOUR hardware.. This finds the bugs you need fixed at a
>| time when people are very receptive to fixing them.
>|
>| It's not realistic for the release engineer to test on a lot
>| of hardware as they are very busy doing other things.
>  =============
>
>Your comment presupposes that most of the bugs are specific to
>one piece of hardware, I doubt that is a valid assertion.  I
>would offer that most of the bugs are not present in source code
>specific to a certain piece of hardware, ...

Some problems are not tied to one specific piece of hardware, but
to the combination of different hardware.  I also went through a
lot of pain with ATA problems for awhile there, and I was fed up
enough that I tried to buy my way out of the problem.  I ended up
with three different SATA controllers, and two different SATA
hard disks.

The thing was, the problems I saw depended on the *combination*
of a hard disk and SATA controller.  My real-SATA hard drive
would fail (in some ways) when connected to one SATA controller,
but not to the other.  And my fake-SATA drive would *work* on
the controller which the real-sata drive failed on, but fail
on the controller the real-sata drive worked on!

There is no question that this was infuriating for me, so I can
sympathize with your frustration.  But I helped Søren get some
hardware he needed for testing, and things gradually improved.
But the problems weren't specific to the hard drive I was using,
or the SATA controller I was using.  They depended on the
combination of pieces that were in my PC.

>Once a bug is reported, and that bug can be reproduced on the
>hardware of the development team, then that bug should not
>reappear again,

In my case, "the development team" needed to *buy* hardware to
reproduce some of the problems I was seeing.  But their hardware
still isn't *exactly* the same as mine.  So, they made some fixes
which solved problems on their hardware and (happily) on mine.
But it is certainly possible for some future change to work
perfectly fine on their hardware, and *not* work on mine.  There
is still no substitute for testing on your hardware, with some
sort of real-world loads.  The project, as such, simply can not
test all combinations of hardware, on all kinds of real-world
loads.  Even if we had a huge collection of PC's to test on,
we're not necessarily going to throw the same kinds of loads
on those machines as you deal with.

I should note that *all* of my SATA-based hardware is stuff that
was not supported at all under 4.x.  So it's awkward for me to
complain too loudly, because I *do* want SATA, and the only way
for FreeBSD to support these new controllers was to make changes
to some previously-working code.

-- 
Garance Alistair Drosehn            =   gad at gilead.netel.rpi.edu
Senior Systems Programmer           or  gad at freebsd.org
Rensselaer Polytechnic Institute    or  drosih at rpi.edu