[RFC] test layout/standardization for FreeBSD

[Poul-Henning Kamp]

--------
In message <F3302426-CA5D-4B61-8F3A-081395B393D9 at xcllnt.net>, Marcel Moolenaar 
writes:

>For example: If a test is not reliable on a heavily loaded machine,
>then the test is ipso facto not 100% deterministic. So, B implies
>A. How is this different from a meta data perspective? If we test
>a RNG, are we ever going to be 100% deterministic?

Some tests are not 100% deterministic no matter what.  That is
an important distinction.

>Also, the estimated duration for tests is very platform specific.

Only if CPU or I/O bound.  Tests for things like dynamic route-expiry,
and other timeout driven parts of the system will have long realtime
clock, but short cpu-clock.

For a comprehensive nightly test-run, you want such test-cases,
for a quick sanitycheck on your kernel rewrite, you probably
don't.

I fully agree with your "crawl before we run" approach, but testing
is a quite mature, if underappreciated, discipline, and we would do
well to learn from others mistakes.

All I'm asking at this point, is that we make a space for storing
this metadata in a machine-readable format.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.