Re: git: 257e70f1d5ee - main - kyua: Add FreeBSD Jail execution environment support

From: Brooks Davis <brooks_at_freebsd.org>
Date: Fri, 19 Jul 2024 21:53:19 UTC
On Fri, Jul 19, 2024 at 02:50:04PM -0700, Ryan Libby wrote:
> On Thu, Jul 18, 2024 at 6:19???AM Mark Johnston <markj@freebsd.org> wrote:
> >
> > The branch main has been updated by markj:
> >
> > URL: https://cgit.FreeBSD.org/src/commit/?id=257e70f1d5ee61037c8c59b116538d3b6b1427a2
> >
> > commit 257e70f1d5ee61037c8c59b116538d3b6b1427a2
> > Author:     Igor Ostapenko <pm@igoro.pro>
> > AuthorDate: 2024-07-16 18:41:12 +0000
> > Commit:     Mark Johnston <markj@FreeBSD.org>
> > CommitDate: 2024-07-18 13:18:28 +0000
> >
> >     kyua: Add FreeBSD Jail execution environment support
> >
> >     A new Kyua concept is added -- "execution environment". A test can be
> >     configured to be run within a specific environment. The test case
> >     lifecycle is extended respectively:
> >     - execenv init (creates a jail or does nothing for default
> >       execenv="host")
> >     - test exec
> >     - cleanup exec (optional)
> >     - execenv cleanup (removes a jail or does nothing for default
> >       execenv="host")
> >
> >     The following new functionality is provided, from bottom to top:
> >
> >     1 ATF based tests
> >
> >     - The new "execenv" metadata property can be set to explicitly ask for
> >       an execution environment: "host" or "jail". If it's not defined, as
> >       all existing tests do, then it implicitly means "host".
> >
> >     - The new "execenv.jail.params" metadata property can be optionally
> >       defined to ask Kyua to use specific jail(8) parameters during creation
> >       of a temporary jail. An example is "vnet allow.raw_sockets".
> >
> >       Kyua implicitly adds "children.max" to "execenv_jail_params"
> >       parameters with the maximum possible value. A test case can override
> >       it.
> >
> >     2 Kyuafile
> >
> >     - The same new metadata properties can be defined on Kyuafile level:
> >       "execenv" and "execenv_jail_params".
> >
> >     - Note that historically ATF uses dotted style of metadata naming, while
> >       Kyua uses underscore style. Hence "execenv.jail.params" vs.
> >       "execenv_jail_params".
> >
> >     3 kyua.conf, kyua CLI
> >
> >     - The new "execenvs" engine configuration variable can be set to a list
> >       of execution environments to run only tests designed for. Tests of not
> >       listed environments are skipped.
> >
> >     - By default, this variable lists all execution environments supported
> >       by a Kyua binary, e.g. execenvs="host jail".
> >
> >     - This variable can be changed via "kyua.conf" or via kyua CLI's "-v"
> >       parameter. For example, "kyua -v execenvs=host test" will run only
> >       host-based tests and skip jail-based ones.
> >
> >     - Current value of this variable can be examined with "kyua config".
> >
> >     [markj] This feature has not landed upstream yet.
> >     See the discussion in https://github.com/freebsd/kyua/pull/224 .
> >     Having the ability to automatically jail tests allows many network tests
> >     to run in parallel, giving a drastic speedup.  So, let's import the
> >     feature and start using it in main.
> >
> >     Signed-off-by:  Igor Ostapenko <pm@igoro.pro>
> >     Reviewed by:    markj, kp
> >     Tested by:      markj, kp
> >     MFC after:      3 months
> >     Differential Revision:  https://reviews.freebsd.org/D45865
> > ---
> >  contrib/kyua/AUTHORS                              |   1 +
> >  contrib/kyua/cli/cmd_config_test.cpp              |  12 +-
> >  contrib/kyua/doc/kyua.conf.5.in                   |  13 +-
> >  contrib/kyua/doc/kyuafile.5.in                    | 103 +++++++-
> >  contrib/kyua/drivers/report_junit_test.cpp        |   8 +
> >  contrib/kyua/engine/atf.cpp                       |  11 +-
> >  contrib/kyua/engine/atf_list.cpp                  |   4 +
> >  contrib/kyua/engine/config.cpp                    |  18 ++
> >  contrib/kyua/engine/execenv/execenv.cpp           |  74 ++++++
> >  contrib/kyua/engine/execenv/execenv.hpp           | 149 +++++++++++
> >  contrib/kyua/engine/execenv/execenv_host.cpp      |  52 ++++
> >  contrib/kyua/engine/execenv/execenv_host.hpp      |  63 +++++
> >  contrib/kyua/engine/plain.cpp                     |   7 +-
> >  contrib/kyua/engine/requirements.cpp              |  33 +++
> >  contrib/kyua/engine/scheduler.cpp                 | 274 ++++++++++++++++++-
> >  contrib/kyua/engine/scheduler.hpp                 |   1 +
> >  contrib/kyua/engine/tap.cpp                       |   7 +-
> >  contrib/kyua/examples/kyua.conf                   |   3 +
> >  contrib/kyua/integration/cmd_config_test.sh       |   2 +
> >  contrib/kyua/integration/cmd_report_junit_test.sh |   8 +
> >  contrib/kyua/integration/cmd_report_test.sh       |   2 +
> >  contrib/kyua/main.cpp                             |   3 +
> >  contrib/kyua/model/metadata.cpp                   |  76 ++++++
> >  contrib/kyua/model/metadata.hpp                   |   5 +
> >  contrib/kyua/model/metadata_test.cpp              |   8 +-
> >  contrib/kyua/model/test_case_test.cpp             |   3 +-
> >  contrib/kyua/model/test_program_test.cpp          |  13 +-
> >  contrib/kyua/os/freebsd/execenv_jail.cpp          |  78 ++++++
> >  contrib/kyua/os/freebsd/execenv_jail.hpp          |  65 +++++
> >  contrib/kyua/os/freebsd/execenv_jail_manager.cpp  |  63 +++++
> >  contrib/kyua/os/freebsd/execenv_jail_manager.hpp  |  54 ++++
> >  contrib/kyua/os/freebsd/execenv_jail_stub.cpp     |  75 ++++++
> >  contrib/kyua/os/freebsd/main.cpp                  |  54 ++++
> >  contrib/kyua/os/freebsd/main.hpp                  |  41 +++
> >  contrib/kyua/os/freebsd/utils/jail.cpp            | 306 ++++++++++++++++++++++
> >  contrib/kyua/os/freebsd/utils/jail.hpp            |  64 +++++
> >  contrib/kyua/utils/config/nodes.ipp               |   9 +-
> >  contrib/kyua/utils/process/executor.cpp           |  42 +++
> >  contrib/kyua/utils/process/executor.hpp           |   1 +
> >  usr.bin/kyua/Makefile                             |  15 +-
> >  40 files changed, 1792 insertions(+), 28 deletions(-)
> >
> 
> This broke the gcc build:
> https://ci.freebsd.org/job/FreeBSD-main-amd64-gcc13_build/1307/

See https://reviews.freebsd.org/D46041 for a workaround.

-- Brooks