From nobody Fri Oct 30 21:59:05 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jGZkf0BnRz6xdSm for ; Fri, 30 Oct 2026 21:59:22 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jGZkd1Jtgz4WmQ for ; Fri, 30 Oct 2026 21:59:21 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20260707 header.b=wy1Cikcy; spf=pass (mx1.freebsd.org: domain of yaneurabeya@gmail.com designates 2607:f8b0:4864:20::633 as permitted sender) smtp.mailfrom=yaneurabeya@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-2e831e86ebfso7542845ad.1 for ; Fri, 30 Oct 2026 14:59:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1793397559; x=1794002359; darn=freebsd.org; h=to:references:message-id:cc:date:in-reply-to:from:subject :mime-version:content-type:from:to:cc:subject:date:message-id :reply-to:content-type; bh=TC97iRXbfA1q61qVCCU9OakQPwqNaApGIldT3XYJYG8=; b=wy1CikcyO/VvPiIw49PhjHDrPQo4wxTUxDBu+LDxhc0EVBlw8723LJ5tQy3Jp05iW5 nsKtY3bOTBDMUYTNMxoAjDMRXveniF75P7v+qaTIACrpnwwvqwJXXsP5U4xDchgEKn/r 6gLKxsZzL7ol8Hj8yO8oUemhwPge7gmrGr4maq/jy0KZs5LvPUwwQVAHYxFWgCOpdUg2 +UjPf46LNK4Be4lL5onY4+oX6hxA69IqTrMh/lJgshim2LL0cuXuc65ap/rK3ieKi+wx e0BLaFzj94HHdq6yOJg9vCQ2szso5ck1bjNqI11nzWHtK6I/XNhkFp1UnSTNYBQH7U7F +srQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1793397559; x=1794002359; h=to:references:message-id:cc:date:in-reply-to:from:subject :mime-version:content-type:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to:content-type; bh=TC97iRXbfA1q61qVCCU9OakQPwqNaApGIldT3XYJYG8=; b=nBLCIfeqKmB7DXIaoOrekwGiSrDovqNEHWOqvWzVPuW9OTHTvZCBv4ezG517tcMPg5 AyIJpqwZxh3t61X1kWT0vH72seLoZ83uBAE19BKboGzH+sxVBXFj/8QVl9KSc52KZcdF VQqUO/KRY/BEOjjm2WhY/WRAcCHYVmhSNcwbtwz366ITbnWNuBJUdfIOdf7EaVGJAMje rmNmQBZi2l3URQP1Bt8Ej23ksQ3+fzBksYxvnvAq07IPKwM81hWZH+wJ+HDViNZLR69s 3sb5gBbhT2lPq3ktiNDG7NHTTAAFHOXLWULxxrEhgSkl52g+s74eMqIzpiQuq3iPHFla eKzQ== X-Forwarded-Encrypted: i=1; AKwUvBzbEbptR1ACNbU8dwjOvDFt1kv+0WOmd+JTZjZw865qW2CLbFdwRrsSWWebchXqQOysC3ZH920Q7TzZsUITC4OlpB1u@freebsd.org X-Gm-Message-State: AFq9FYIwEusGRCQ7NXghcU957ftNF4Fl+TUzaJYbMMDAHDVK09Rz8WCl 5QRjjcFDupA7tn1EXyWoZcDMpC8aeAIZ9nN59eqm17rJFfUwNAG32fo4 X-Gm-Gg: AYBFou1UOyyvi0RLkuEvQodAvblbqZhZdA33NDaEyMvN+ECnV8Ql90CpCEWykHwMwQu RbU+1LMel+dZE0DdE24ubs9659vOHYr68kD+EPJ87U3bZVRmBGcqi2f9cDU/m7PVTVby+HD5cmH c3t7Fnh6hLa6W/nPHAkmFn6LtmcrJ9WNASgGOzGeyVs4EPRRcnxk0eRCNB0LQsiSbqgRHVpdTQE Irm0Q+b8d0dihEq5Oi5gSzDw1oLvo0LlczF+CNWGmm6dtOdAHeefXQZ7dOL1GMu300PiQgy1HPh K6WX0t25f878l+fYPNFwq+ut6aRNwcafFXZ1Ra4XYY8l7SRdtAD1tdlLwb41aBQFtu+K09fsNaX uO2egSGcS+EkEn4s7PRUiC43GDEtZRVrI5TWihb6RVgPxzMGsjHPy5Poj1nHnvJjRRaRq0mCTsi HSJeS+VOpcKsX4qCLXNTWNTbdTciCdvopcoD+LneWr09+RWBtr+hUgkDbiVWTt7K9h3ijc0MMnu IMKjQo0KRfkHyBkUQDB06P3tGwD4ZkzHXy5TOUHCj1GWXwZAsVsLQTlCb24RCwSVg8KALzmp/4N E7Tz/Qc9eMR0VzNcWJQV5Ezo688iGy+1RBrU6WJPrYiaTD4P5ifz X-Received: by 2002:a17:90b:5648:b0:3ad:93b4:718f with SMTP id 98e67ed59e1d1-3ae89879b52mr3262338a91.65.1793397558512; Fri, 30 Oct 2026 14:59:18 -0700 (PDT) Received: from smtpclient.apple ([187.15.154.221]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-cdb09813a37sm1222011a12.23.2026.10.30.14.59.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 30 Oct 2026 14:59:17 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_379BEEEE-10E4-4F5E-BD7C-985CCC0B8A74"; protocol="application/pgp-signature"; micalg=pgp-sha256 List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.700.51.1.3\)) Subject: Re: git: 3fb03fb35b20 - main - [tests] switch GoogleTest tests to `googletest` interface From: "Enji Cooper (yaneurabeya)" In-Reply-To: Date: Fri, 30 Oct 2026 14:59:05 -0700 Cc: Enji Cooper , src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Message-Id: <89E29294-CBC3-4047-BDE1-8BF9BD978A09@gmail.com> References: <6ae3d7a5.39b6a.63003bda@gitrepo.freebsd.org> <15E8ABE9-8E0C-4950-983A-FC429826890A@gmail.com> To: Mark Johnston X-Mailer: Apple Mail (2.3864.700.51.1.3) X-Rspamd-Queue-Id: 4jGZkd1Jtgz4WmQ X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.60 / 15.00]; SIGNED_PGP(-2.00)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4864::/56]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20260707]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::633:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; PREVIOUSLY_DELIVERED(0.00)[dev-commits-src-all@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[dev-commits-src-all@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] --Apple-Mail=_379BEEEE-10E4-4F5E-BD7C-985CCC0B8A74 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Oct 30, 2026, at 2:43=E2=80=AFPM, Mark Johnston = wrote: >=20 > On Fri, Oct 30, 2026 at 01:54:22PM -0700, Enji Cooper (yaneurabeya) = wrote: >>=20 >>> On Oct 29, 2026, at 7:20=E2=80=AFPM, Mark Johnston = wrote: >>>=20 >>> On Thu, Oct 29, 2026 at 11:32:53PM +0000, Enji Cooper wrote: >>>> The branch main has been updated by ngie: >>>>=20 >>>> URL: = https://cgit.FreeBSD.org/src/commit/?id=3D3fb03fb35b20d259e0a3a07fc48f2ecc= 3c5da6e8 >>>>=20 >>>> commit 3fb03fb35b20d259e0a3a07fc48f2ecc3c5da6e8 >>>> Author: Enji Cooper >>>> AuthorDate: 2026-10-07 18:53:15 +0000 >>>> Commit: Enji Cooper >>>> CommitDate: 2026-10-29 23:32:11 +0000 >>>>=20 >>>> [tests] switch GoogleTest tests to `googletest` interface >>>=20 >>> Please follow the guidelines for commit log messages: >>> = https://docs.freebsd.org/en/articles/committers-guide/#_prefix_the_subject= _line_with_a_component_if_applicable >>>=20 >>>> The new `googletest` kyua test interface--added in 0.15.0 and = fixed in >>>> 0.15.1--allows tests to be executed and handled in a more = fine-grained >>>> manner than the way it was prior to this commit. >>>>=20 >>>> This change deletes the `capsicum-test` ATF wrapper script, >>>> `functional.sh`, and all supporting logic in `googletest.test.mk` = as >>>> well as this change makes that logic unnecessary. >>>>=20 >>>> MFC after: 1 month >>>> MFC with: b68969979d6b ("contrib/kyua: update to 0.15.1") >>>> --- >>>> share/mk/googletest.test.mk | 11 +------- >>>> tests/sys/capsicum/Makefile | 1 - >>>> tests/sys/capsicum/functional.sh | 60 = ---------------------------------------- >>>> 3 files changed, 1 insertion(+), 71 deletions(-) >>>>=20 >>>> diff --git a/share/mk/googletest.test.mk = b/share/mk/googletest.test.mk >>>> index 026f6dcacce3..b21576112a78 100644 >>>> --- a/share/mk/googletest.test.mk >>>> +++ b/share/mk/googletest.test.mk >>>> @@ -6,10 +6,6 @@ >>>> # GoogleTest is a C++ test framework, thus, it does not = describe/articulate how >>>> # to write tests in other languages, e.g., C or shell, unlike the = ATF, plain, >>>> # and TAP raw test interfaces. >>>> -# >>>> -# For now this is a thin wrapper around the `plain` test = interface, but in the >>>> -# future this will rely on a newer version of kyua which will = integrate in >>>> -# GoogleTest support. >>>>=20 >>>> .if !target(____) >>>> .error googletest.test.mk cannot be included directly. >>>> @@ -34,13 +30,8 @@ BINDIR.${_T}=3D ${TESTSDIR} >>>> CXXFLAGS.${_T}+=3D ${GTESTS_CXXFLAGS} >>>> MAN.${_T}?=3D # empty >>>> SRCS.${_T}?=3D ${_T}.cc >>>> -.if !empty(GTESTS_WRAPPER_SH.${_T}) >>>> -# A stopgap/workaround to let kyua execute test case one by one >>>> -ATF_TESTS_SH+=3D ${GTESTS_WRAPPER_SH.${_T}} >>>> -.else >>>> _TESTS+=3D ${_T} >>>> -TEST_INTERFACE.${_T}=3D plain >>>> -.endif >>>> +TEST_INTERFACE.${_T}=3D googletest >>>>=20 >>>> .endfor >>>> .endif >>>> diff --git a/tests/sys/capsicum/Makefile = b/tests/sys/capsicum/Makefile >>>> index a8f72d399af9..788fa9f9394c 100644 >>>> --- a/tests/sys/capsicum/Makefile >>>> +++ b/tests/sys/capsicum/Makefile >>>> @@ -10,7 +10,6 @@ CFLAGS+=3D -I${SRCTOP}/tests >>>> .if ${MK_GOOGLETEST} !=3D no >>>>=20 >>>> GTESTS+=3D capsicum-test >>>> -GTESTS_WRAPPER_SH.capsicum-test=3D functional >>>> # This test script runs the same test suite twice, once as root and = once as an >>>> # unprivileged user. Serialize them since some tests access global = namespaces, >>>> # e.g., mqueuefs, and can trample on each other. >>>=20 >>> Below this there are some TEST_METADATA.functional configuration >>> variables still set. >>=20 >> I=E2=80=99ll take a look at this. >>=20 >>> After this commit I see a whole bunch of newly skipped tests, it = looks >>> like a regression: >>=20 >> =E2=80=A6 >>=20 >> It=E2=80=99s not. The problem is that prerequisites aren=E2=80=99t = being met or the test cases were skipped previously, but not counted as = skipped. >>=20 >> Prior to this change all GoogleTest tests were run as =E2=80=9Cplain=E2= =80=9D interface test programs, which had a binary notion of pass/fail: = did the program exit with 0? If so the test passed, else it failed. >>=20 >> Using ` = lib/googletest/gtest_main/googletest-printers-test:PrintStlContainerTest.S= tdSpan` as an example. Before (from = https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29832/testReport/lib.go= ogletest.gtest_main/googletest-printers-test/ ): >> ``` >> [ SKIPPED ] 1 test, listed below: >> [ SKIPPED ] PrintStlContainerTest.StdSpan >> ``` >> The testcase was being skipped - the suite as a whole exited as 0, so = the suite =E2=80=9Cpassed=E2=80=9D instead of "failed=E2=80=9D. >>=20 >> sys.capsicum.capsicum-test: some of the tests were skipped because = they weren=E2=80=99t run as root. For example: = https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/sys.ca= psicum/capsicum-test/ . Prior to this change the tests were run both as = root and as an unprivileged user - this is =E2=80=9Ceasy to fix=E2=80=9D = by linking the test and adding the appropriate TEST_METADATA to it = (it=E2=80=99s a minor regression). >>=20 >> sys/fs/fusefs/default_permissions:Access.eacces passed in CI... = https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/junit/= sys.fs.fusefs/default_permissions/Access_eacces/ =E2=80=A6 along most of = the other fusefs tests: a few are skipped, but are explicitly marked = =E2=80=9CDISABLED=E2=80=9D, so their prereqs haven=E2=80=99t been met = yet per the test: = https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29845/testReport/sys.fs= .fusefs/write/AioWrite_DISABLED_aio_write/ (some of the tests skipped in = CI are lacking =E2=80=9Creasons=E2=80=9D for why the tests are = disabled). >>=20 >> Are you running the test locally? If so, please check the =E2=80=9Cskip= =E2=80=9D message (there are a few prereqs that need to be fulfilled in = order to run the tests). >=20 > Yes I'm running them locally. It turns out the fusefs tests were > skipping because vfs.usermount is set to 0 there; the message from the > tests is just "current user is not allowed to mount" in that case. So > that's okay. >=20 > The capsicum tests are skipping because they're run with > required_user=3D"unprivileged" but a few of them require root = privileges, > so there is indeed some regression from commit > 3fb03fb35b20d259e0a3a07fc48f2ecc3c5da6e8. Well, some good news and bad news on this front. Good news: the tests = will be run as root again as well as unprivileged users (diff pending), = restoring test coverage. Bad news is that there will be =E2=80=9Ctest = result bloat=E2=80=9D: there=E2=80=99s no way to really get around the = fact that a ton of tests when run as an unprivileged user will be = skipped with the same message (=E2=80=9Crequires root=E2=80=9D). This = can be fixed with some light refactoring, but it=E2=80=99s not a simple = Makefile change like the (soon-to-be) proposed change will be. Some of the other permutations previously allowed with capsicum-test = available in the larger test suite might need to be added (there are = additional arguments which support different behavior in = `capsicum-test-main.cc`), but this isn=E2=80=99t a regression since this = was the way things were prior to = 3fb03fb35b20d259e0a3a07fc48f2ecc3c5da6e8 . The only real net-change in = behavior in the tests is that functional.sh explicitly tested root and = non-root users and functional.sh ran the entire test suite, unlike kyua = does with googletest now (kyua runs each testcase in isolation, = process-wise). -Enji= --Apple-Mail=_379BEEEE-10E4-4F5E-BD7C-985CCC0B8A74 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEkHfexGRJ3gYRdA2gGpE5DjPsNJgFAmrlEykACgkQGpE5DjPs NJjXzQ//dexg3SFx7jXQ/gfEbmKNkmBoGSX+l/eTMxlYDBTUvh99kSXkW4PmMu9o O+caUONeQgnbeUHVBqF2Qv2XEf0RYamHD44cSWJpV+yBoF01fP7RPdjqCkUpEHEt jPbpxUIU53lLL08Hk+UNWKJIUT1mAsRXoBKMgbuf79LGycf2har4VHqEe3Q0Mht+ 2uyeZfNyMKexms1bVBwinZlCiLUcOfUyNyiDz1oBmyWUCIcespYsXT+noqeSFx05 4ZJy+ChaN3FPQ1AuGwkP5JAlgyNj22/ffjUqsOWOqxDM4QJd9FK6KDM6iSiRca0W /wA/Su9D7RvU8stQySQe+hoU9rjOWaC4NG00lBJTOYjZgkJT4QzCvjuOYbeWtOOI FNxM83VAPtcVxqlWxQxHCvlss5V7plChSyoU2Gu/ZoAHsFne3VzjtDtyq+4YL2RK hZvAWg25PSFmdThIiNdQPT5dxjoZKHmVeoHJepMq/YKefX0aDU432l/0oGMgnakn q2y2imYlQS8oaV6Jdpm1f9imFacfh1md/rT8t5jVqGd9lT8zOSCGv43Ec5IEU+SU eVkopasuHxbijmrLNp6E1/AwAgBtqrodmMdkWCyOXikRfqCxtZPuS2t6QI+vkSgc 4WBeL1L9ZPgwDXW6/oQX8E/gqlcaFyioU0GdWV4mq6XIRvyMnvc= =q59m -----END PGP SIGNATURE----- --Apple-Mail=_379BEEEE-10E4-4F5E-BD7C-985CCC0B8A74--