From nobody Fri Oct 30 21:43:09 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jGZMz71BGz6xchY; Fri, 30 Oct 2026 21:43:11 +0000 (UTC) (envelope-from markj@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jGZMz6F3rz4TMT; Fri, 30 Oct 2026 21:43:11 +0000 (UTC) (envelope-from markj@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793396591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7MIqqDWp79JYQYu4zJkx1PwB+DI0HNQc7rVZ1yT98QU=; b=agDWXTNjR8dFg8ECK04DT9ouugN8oNmcrUSlSTluJRVcBzReUfySfR2c0Z+MPCXF0NRzU1 h5LDMeiVNvaiNiAxUOsK/IO4P6syKf65tWHtV6kYw4keP6O40LpLAM258nA80iXguGkANE GVpk+BHny/SR5dkqK5Pt8al/3qjqWnQZYkAGaAkK+Mig/3q0b2ANg378IwEnV8drXxGEsl c57yIqEcv7y6CdPPin/ZkvRk0obxHDKgO15a/5tzftUqmbLCyP0EHaHtoLa8aKsoHo3Bp7 dX6cU6pq35pNliucektiR++4TKTnjh6u3FFD4uCq2hT7X8INN4TlzkBr5U841A== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793396591; b=PtTApz8nKzoawSktczPw5ObrF+hmOVoO+qci+4Uak/daeGOFLggH6E7irLq5NpD6t0MxeP HJC7vmERyaJ7zXrQmAZfVgKTZMLMusA5O/0lVabTzrAI8iOa27eTwNUcxIZLIhtXe9NDyS iiUqUeO6H0rQlqKubBKQsuMiDwWMVM62MD3ncDcaQmBMmTxigEdRiGom2EkxqL3zUq2gv/ YHy4gIKI6LohX8FAb2+P0ztRmpa34ItIBSIMFrcuQ/zMYhnVLyGz4ATmgdBKaKw6UXgj03 oVU2TUH0k3+EhdTJ0UkxyvdKJydZspRvvfZHoLRToja3N50JxiTQWsXHKg9UYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793396591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7MIqqDWp79JYQYu4zJkx1PwB+DI0HNQc7rVZ1yT98QU=; b=vpuY6p3WJmBkSYr4uak/MTrfiW/4N9eER/uFBztgaI359AvFiyo+eLMK1xKPi4wlQHGvpk kZyNWxWGREYM/HwI1Jork3fz2V4Y6BRTci9FxxMHcwuvyWIxPz8otY1EUhqIWed+3tXdKb HewRKHlPnsPDQgV21GSXP6ftGYreGwV0lOzPzT1lY9VtsSlICArpS9dDTqQzcSQ8VrJ4mv 6gcxX0m1k7dbXA7Q39pRmLPU/AzxTaGk2LUYY9XuVmz74ycQtWAV7cdhu5/7KWvomuzLGa hH9QQ4IXUwJS3Zt5+jxSwrxwIQATHcq4MR+1bkNk1ATCoujeQ0xbYiO66G2zCQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from nuc (198-84-224-249.cpe.teksavvy.com [198.84.224.249]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: markj) by smtp.freebsd.org (Postfix) with ESMTPSA id 4jGZMz2nlzz17bs; Fri, 30 Oct 2026 21:43:11 +0000 (UTC) (envelope-from markj@freebsd.org) Date: Fri, 30 Oct 2026 17:43:09 -0400 From: Mark Johnston To: "Enji Cooper (yaneurabeya)" Cc: Enji Cooper , src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 3fb03fb35b20 - main - [tests] switch GoogleTest tests to `googletest` interface Message-ID: References: <6ae3d7a5.39b6a.63003bda@gitrepo.freebsd.org> <15E8ABE9-8E0C-4950-983A-FC429826890A@gmail.com> List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <15E8ABE9-8E0C-4950-983A-FC429826890A@gmail.com> On Fri, Oct 30, 2026 at 01:54:22PM -0700, Enji Cooper (yaneurabeya) wrote: > > > On Oct 29, 2026, at 7:20 PM, Mark Johnston wrote: > > > > On Thu, Oct 29, 2026 at 11:32:53PM +0000, Enji Cooper wrote: > >> The branch main has been updated by ngie: > >> > >> URL: https://cgit.FreeBSD.org/src/commit/?id=3fb03fb35b20d259e0a3a07fc48f2ecc3c5da6e8 > >> > >> commit 3fb03fb35b20d259e0a3a07fc48f2ecc3c5da6e8 > >> Author: Enji Cooper > >> AuthorDate: 2026-10-07 18:53:15 +0000 > >> Commit: Enji Cooper > >> CommitDate: 2026-10-29 23:32:11 +0000 > >> > >> [tests] switch GoogleTest tests to `googletest` interface > > > > Please follow the guidelines for commit log messages: > > https://docs.freebsd.org/en/articles/committers-guide/#_prefix_the_subject_line_with_a_component_if_applicable > > > >> The new `googletest` kyua test interface--added in 0.15.0 and fixed in > >> 0.15.1--allows tests to be executed and handled in a more fine-grained > >> manner than the way it was prior to this commit. > >> > >> This change deletes the `capsicum-test` ATF wrapper script, > >> `functional.sh`, and all supporting logic in `googletest.test.mk` as > >> well as this change makes that logic unnecessary. > >> > >> MFC after: 1 month > >> MFC with: b68969979d6b ("contrib/kyua: update to 0.15.1") > >> --- > >> share/mk/googletest.test.mk | 11 +------- > >> tests/sys/capsicum/Makefile | 1 - > >> tests/sys/capsicum/functional.sh | 60 ---------------------------------------- > >> 3 files changed, 1 insertion(+), 71 deletions(-) > >> > >> diff --git a/share/mk/googletest.test.mk b/share/mk/googletest.test.mk > >> index 026f6dcacce3..b21576112a78 100644 > >> --- a/share/mk/googletest.test.mk > >> +++ b/share/mk/googletest.test.mk > >> @@ -6,10 +6,6 @@ > >> # GoogleTest is a C++ test framework, thus, it does not describe/articulate how > >> # to write tests in other languages, e.g., C or shell, unlike the ATF, plain, > >> # and TAP raw test interfaces. > >> -# > >> -# For now this is a thin wrapper around the `plain` test interface, but in the > >> -# future this will rely on a newer version of kyua which will integrate in > >> -# GoogleTest support. > >> > >> .if !target(____) > >> .error googletest.test.mk cannot be included directly. > >> @@ -34,13 +30,8 @@ BINDIR.${_T}= ${TESTSDIR} > >> CXXFLAGS.${_T}+= ${GTESTS_CXXFLAGS} > >> MAN.${_T}?= # empty > >> SRCS.${_T}?= ${_T}.cc > >> -.if !empty(GTESTS_WRAPPER_SH.${_T}) > >> -# A stopgap/workaround to let kyua execute test case one by one > >> -ATF_TESTS_SH+= ${GTESTS_WRAPPER_SH.${_T}} > >> -.else > >> _TESTS+= ${_T} > >> -TEST_INTERFACE.${_T}= plain > >> -.endif > >> +TEST_INTERFACE.${_T}= googletest > >> > >> .endfor > >> .endif > >> diff --git a/tests/sys/capsicum/Makefile b/tests/sys/capsicum/Makefile > >> index a8f72d399af9..788fa9f9394c 100644 > >> --- a/tests/sys/capsicum/Makefile > >> +++ b/tests/sys/capsicum/Makefile > >> @@ -10,7 +10,6 @@ CFLAGS+= -I${SRCTOP}/tests > >> .if ${MK_GOOGLETEST} != no > >> > >> GTESTS+= capsicum-test > >> -GTESTS_WRAPPER_SH.capsicum-test= functional > >> # This test script runs the same test suite twice, once as root and once as an > >> # unprivileged user. Serialize them since some tests access global namespaces, > >> # e.g., mqueuefs, and can trample on each other. > > > > Below this there are some TEST_METADATA.functional configuration > > variables still set. > > I’ll take a look at this. > > > After this commit I see a whole bunch of newly skipped tests, it looks > > like a regression: > > … > > It’s not. The problem is that prerequisites aren’t being met or the test cases were skipped previously, but not counted as skipped. > > Prior to this change all GoogleTest tests were run as “plain” interface test programs, which had a binary notion of pass/fail: did the program exit with 0? If so the test passed, else it failed. > > Using ` lib/googletest/gtest_main/googletest-printers-test:PrintStlContainerTest.StdSpan` as an example. Before (from https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29832/testReport/lib.googletest.gtest_main/googletest-printers-test/ ): > ``` > [ SKIPPED ] 1 test, listed below: > [ SKIPPED ] PrintStlContainerTest.StdSpan > ``` > The testcase was being skipped - the suite as a whole exited as 0, so the suite “passed” instead of "failed”. > > sys.capsicum.capsicum-test: some of the tests were skipped because they weren’t run as root. For example: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/sys.capsicum/capsicum-test/ . Prior to this change the tests were run both as root and as an unprivileged user - this is “easy to fix” by linking the test and adding the appropriate TEST_METADATA to it (it’s a minor regression). > > sys/fs/fusefs/default_permissions:Access.eacces passed in CI... https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/junit/sys.fs.fusefs/default_permissions/Access_eacces/ … along most of the other fusefs tests: a few are skipped, but are explicitly marked “DISABLED”, so their prereqs haven’t been met yet per the test: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/sys.fs.fusefs/write/AioWrite_DISABLED_aio_write/ (some of the tests skipped in CI are lacking “reasons” for why the tests are disabled). > > Are you running the test locally? If so, please check the “skip” message (there are a few prereqs that need to be fulfilled in order to run the tests). Yes I'm running them locally. It turns out the fusefs tests were skipping because vfs.usermount is set to 0 there; the message from the tests is just "current user is not allowed to mount" in that case. So that's okay. The capsicum tests are skipping because they're run with required_user="unprivileged" but a few of them require root privileges, so there is indeed some regression from commit 3fb03fb35b20d259e0a3a07fc48f2ecc3c5da6e8.