Skipping tests that are unimplemented in 32-bit emulation

[Mark Johnston]

On Tue, Aug 07, 2018 at 10:11:45AM -0600, Alan Somers wrote:
> On Tue, Aug 7, 2018 at 9:24 AM, Mark Johnston <markj at freebsd.org> wrote:
> > Oops, right.  For 2), then, couldn't we define test_suite("FreeBSD32")
> > in the kyuafiles for the 32-bit test suite and set a
> > test_suite.FreeBSD32.compat32=true variable in kyua.conf?  Then a test
> > which uses, say, setfib(1) can query that variable and skip if it's set.
> >
> 
> Ok, but what will set the test_suite.FreeBSD32.compat32 variable?  It can't
> be set as part of the image build, because then it will have the wrong
> value in jails and chroots.  It needs to be set at runtime.

For my suggestion to work, the 32-bit test suite must be built with
TESTSUITE=FreeBSD32 instead of the default in suite.test.mk.  This could
be accomplished in a few ways:
1) Modify the amd64 build to additionally build a 32-bit copy of the
   test suite with TESTSUITE=FreeBSD32 and install it to /usr/tests32.
   Users that wish to run it would have to copy it into their
   chroot/jail.
2) When building a 32-bit image for a chroot/jail, just remember to
   specify TESTSUITE=FreeBSD32 in the make(1) invocation.  (I didn't
   actually verify that this does what I expect.)
3) Always define the test suite name to be FreeBSD32 or FreeBSD64
   depending on the target architecture.  Then, tests that need to be
   skipped when run under COMPAT32 would need to check for the
   "compat32" variable _and_ determine if the host kernel is 64-bit
   (e.g., by looking at the hw.machine sysctl).

If you do 2) and just set test_suite.FreeBSD32.compat32 statically in
/usr/local/etc/kyua.conf, then I don't see why you need any special
runtime logic.  Any tests that aren't supported by COMPAT32 can just be
skipped when the compat32 kyua variable is set.