Re: git: 3fb03fb35b20 - main - [tests] switch GoogleTest tests to `googletest` interface

> On Oct 29, 2026, at 7:20 PM, Mark Johnston <markj@FreeBSD.org> wrote:
> 
> On Thu, Oct 29, 2026 at 11:32:53PM +0000, Enji Cooper wrote:
>> The branch main has been updated by ngie:
>> 
>> URL: https://cgit.FreeBSD.org/src/commit/?id=3fb03fb35b20d259e0a3a07fc48f2ecc3c5da6e8
>> 
>> commit 3fb03fb35b20d259e0a3a07fc48f2ecc3c5da6e8
>> Author:     Enji Cooper <ngie@FreeBSD.org>
>> AuthorDate: 2026-10-07 18:53:15 +0000
>> Commit:     Enji Cooper <ngie@FreeBSD.org>
>> CommitDate: 2026-10-29 23:32:11 +0000
>> 
>>    [tests] switch GoogleTest tests to `googletest` interface
> 
> Please follow the guidelines for commit log messages:
> https://docs.freebsd.org/en/articles/committers-guide/#_prefix_the_subject_line_with_a_component_if_applicable
> 
>>    The new `googletest` kyua test interface--added in 0.15.0 and fixed in
>>    0.15.1--allows tests to be executed and handled in a more fine-grained
>>    manner than the way it was prior to this commit.
>> 
>>    This change deletes the `capsicum-test` ATF wrapper script,
>>    `functional.sh`, and all supporting logic in `googletest.test.mk` as
>>    well as this change makes that logic unnecessary.
>> 
>>    MFC after:      1 month
>>    MFC with:       b68969979d6b ("contrib/kyua: update to 0.15.1")
>> ---
>> share/mk/googletest.test.mk      | 11 +-------
>> tests/sys/capsicum/Makefile      |  1 -
>> tests/sys/capsicum/functional.sh | 60 ----------------------------------------
>> 3 files changed, 1 insertion(+), 71 deletions(-)
>> 
>> diff --git a/share/mk/googletest.test.mk b/share/mk/googletest.test.mk
>> index 026f6dcacce3..b21576112a78 100644
>> --- a/share/mk/googletest.test.mk
>> +++ b/share/mk/googletest.test.mk
>> @@ -6,10 +6,6 @@
>> # GoogleTest is a C++ test framework, thus, it does not describe/articulate how
>> # to write tests in other languages, e.g., C or shell, unlike the ATF, plain,
>> # and TAP raw test interfaces.
>> -#
>> -# For now this is a thin wrapper around the `plain` test interface, but in the
>> -# future this will rely on a newer version of kyua which will integrate in
>> -# GoogleTest support.
>> 
>> .if !target(__<bsd.test.mk>__)
>> .error googletest.test.mk cannot be included directly.
>> @@ -34,13 +30,8 @@ BINDIR.${_T}= ${TESTSDIR}
>> CXXFLAGS.${_T}+= ${GTESTS_CXXFLAGS}
>> MAN.${_T}?= # empty
>> SRCS.${_T}?= ${_T}.cc
>> -.if !empty(GTESTS_WRAPPER_SH.${_T})
>> -# A stopgap/workaround to let kyua execute test case one by one
>> -ATF_TESTS_SH+= ${GTESTS_WRAPPER_SH.${_T}}
>> -.else
>> _TESTS+= ${_T}
>> -TEST_INTERFACE.${_T}= plain
>> -.endif
>> +TEST_INTERFACE.${_T}= googletest
>> 
>> .endfor
>> .endif
>> diff --git a/tests/sys/capsicum/Makefile b/tests/sys/capsicum/Makefile
>> index a8f72d399af9..788fa9f9394c 100644
>> --- a/tests/sys/capsicum/Makefile
>> +++ b/tests/sys/capsicum/Makefile
>> @@ -10,7 +10,6 @@ CFLAGS+= -I${SRCTOP}/tests
>> .if ${MK_GOOGLETEST} != no
>> 
>> GTESTS+= capsicum-test
>> -GTESTS_WRAPPER_SH.capsicum-test= functional
>> # This test script runs the same test suite twice, once as root and once as an
>> # unprivileged user.  Serialize them since some tests access global namespaces,
>> # e.g., mqueuefs, and can trample on each other.
> 
> Below this there are some TEST_METADATA.functional configuration
> variables still set.

I’ll take a look at this.

> After this commit I see a whole bunch of newly skipped tests, it looks
> like a regression:

…

It’s not. The problem is that prerequisites aren’t being met or the test cases were skipped previously, but not counted as skipped.

Prior to this change all GoogleTest tests were run as “plain” interface test programs, which had a binary notion of pass/fail: did the program exit with 0? If so the test passed, else it failed.

Using ` lib/googletest/gtest_main/googletest-printers-test:PrintStlContainerTest.StdSpan` as an example. Before (from https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29832/testReport/lib.googletest.gtest_main/googletest-printers-test/ ):
```
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] PrintStlContainerTest.StdSpan
```
The testcase was being skipped - the suite as a whole exited as 0, so the suite “passed” instead of "failed”.

sys.capsicum.capsicum-test: some of the tests were skipped because they weren’t run as root. For example: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/sys.capsicum/capsicum-test/ . Prior to this change the tests were run both as root and as an unprivileged user - this is “easy to fix” by linking the test and adding the appropriate TEST_METADATA to it (it’s a minor regression).

sys/fs/fusefs/default_permissions:Access.eacces passed in CI... https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/junit/sys.fs.fusefs/default_permissions/Access_eacces/ … along most of the other fusefs tests: a few are skipped, but are explicitly marked “DISABLED”, so their prereqs haven’t been met yet per the test: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/sys.fs.fusefs/write/AioWrite_DISABLED_aio_write/ (some of the tests skipped in CI are lacking “reasons” for why the tests are disabled).

Are you running the test locally? If so, please check the “skip” message (there are a few prereqs that need to be fulfilled in order to run the tests).

Thanks!
-Enji