Incorporating the Capsicum test suite

Ed Maste emaste at freebsd.org
Tue Jun 2 20:40:55 UTC 2015


I would like to incorporate the Capsicum test suite[1] into our test
infrastructure and Jenkins. There are a small handful of minor issues
to resolve, but it's very close to a state where it "just builds and
passes."

It uses googletest, which produces output of the form:

[----------] 10 tests from Capmode
[ RUN      ] Capmode.AllowedIdentifierSyscallsForked
[       OK ] Capmode.AllowedIdentifierSyscallsForked (11 ms)
[ RUN      ] Capmode.AllowedSchedSyscallsForked
capmode.cc:289: Failure
Expected: (0) <= (sched_rr_get_interval(0, &ts)), actual: 0 vs -1
   errno 94 Not permitted in capability mode
capmode.cc:276: Failure
Value of: rc
  Actual: 1
Expected: 0
[  FAILED  ] Capmode.AllowedSchedSyscallsForked (11 ms)

and reports success/failure via the result code.

I have a couple of questions. First, do we have an existing pattern
for connecting googletest tests to Kyua? We could incorporate it
fairly easily by just encapsulating the whole test, but integrating at
the level of individual tests would be much better.

Second, I wonder if we should import these tests into contrib/ and
build them during buildworld, or put them into a port and install them
into /usr/local/tests/capsicum. I'd lean towards putting them in
contrib since Capsicum is an integrated part of the base system, but
it might be a bit easier to maintain if we keep it as a port/package.
Presumably the Jenkins run will just execute all tests in /usr/tests/
and /usr/local/tests/.

[1] https://github.com/google/capsicum-test


More information about the freebsd-testing mailing list