ATF config variables for FreeBSD tests

[Alan Somers]

Another thread recently raised the issue of tests that cause
potentially harmful changes to sysctl variables.  We all agreed to
conditionalize that test based on an ATF config variable.  Now Ken and
I both want to add tests with different harmful side effects.  I
propose the following list of config variables.  If they sound good to
everyone, I'll add them to tests(7) and to an example kyua.conf file
in devel/kyua.

test_suites.FreeBSD.allow_sysctl_side_effects
If defined, enables tests that cause potentially harmful changes to
globally relevant sysctl variables.  Requested by Peter for
kern_descrip_test:maxfiles__increase

test_suites.FreeBSD.fibs
Defined to a space delimited list of fib numbers.  Tests that change
routing tables must use these fibs instead of the default.  They will
be skipped if the variable is undefined.  I'm working on such tests
right now.

test_suites.FreeBSD.disks
Defined to a space delimited list of disk device nodes.  Tests that
format, mount, write to, disable, or have any other harmful effect on
disks must use these devices.

test_suites.FreeBSD.allow_devfs_side_effects.
If defined, enables tests that can create or destroy semipermanent
device nodes, like disk devices.  Without this variable, tests may
still create and destroy device nodes that are normally transient,
like /dev/tap* and /dev/pts/*.  This variable requires further
explanation:

Naively, one would think that test_suites.FreeBSD.disks would be
sufficient protection for tests that create and destroy disk device
nodes.  However, Ken is working on a test that will cause hard drives
to disappear and reappear in a different order, possibly resulting in
their devnames changing.  It could even result in "holes" appearing in
the list of da unit numbers.  If the sysadmin defines
test_suites.FreeBSD.disks to "/dev/da1 /dev/da2 /dev/da3", it would be
possible for those same devices to be named "/dev/da2 /dev/da1
/dev/da4" by the time Ken's test finishes.  The permanent
disappearance of /dev/da3 could confuse future tests that rely on
test_suites.FreeBSD.disks.  They would try to open /dev/da3 and fail.
Thus, it may not be safe to run Ken's test in the same run as other
tests that access disks.

test_suites.FreeBSD.unit_tests
test_suites.FreeBSD.functional_tests
test_suites.FreeBSD.stress_tests
These three variables would address a different concern of Peter's.
He was concerned that his test might require too much run time,
annoying other users of the rest of the test suite.  By using these
variables, a sysadmin could easily disable stress tests, which tend to
take a very long time to run.  Conversely, unit tests are normally
very fast.  A kyua run that only ran unit tests could easily be done
after every buildworld, for example, while one that ran functional
tests as well could be done perhaps several times per day.  This is
just an idea I have.  I haven't tried it in practice.

What do you think?

-Alan