Re: git: 257e70f1d5ee - main - kyua: Add FreeBSD Jail execution environment support

From: Ryan Libby <rlibby_at_freebsd.org>
Date: Fri, 19 Jul 2024 21:50:04 UTC
On Thu, Jul 18, 2024 at 6:19 AM Mark Johnston <markj@freebsd.org> wrote:
>
> The branch main has been updated by markj:
>
> URL: https://cgit.FreeBSD.org/src/commit/?id=257e70f1d5ee61037c8c59b116538d3b6b1427a2
>
> commit 257e70f1d5ee61037c8c59b116538d3b6b1427a2
> Author:     Igor Ostapenko <pm@igoro.pro>
> AuthorDate: 2024-07-16 18:41:12 +0000
> Commit:     Mark Johnston <markj@FreeBSD.org>
> CommitDate: 2024-07-18 13:18:28 +0000
>
>     kyua: Add FreeBSD Jail execution environment support
>
>     A new Kyua concept is added -- "execution environment". A test can be
>     configured to be run within a specific environment. The test case
>     lifecycle is extended respectively:
>     - execenv init (creates a jail or does nothing for default
>       execenv="host")
>     - test exec
>     - cleanup exec (optional)
>     - execenv cleanup (removes a jail or does nothing for default
>       execenv="host")
>
>     The following new functionality is provided, from bottom to top:
>
>     1 ATF based tests
>
>     - The new "execenv" metadata property can be set to explicitly ask for
>       an execution environment: "host" or "jail". If it's not defined, as
>       all existing tests do, then it implicitly means "host".
>
>     - The new "execenv.jail.params" metadata property can be optionally
>       defined to ask Kyua to use specific jail(8) parameters during creation
>       of a temporary jail. An example is "vnet allow.raw_sockets".
>
>       Kyua implicitly adds "children.max" to "execenv_jail_params"
>       parameters with the maximum possible value. A test case can override
>       it.
>
>     2 Kyuafile
>
>     - The same new metadata properties can be defined on Kyuafile level:
>       "execenv" and "execenv_jail_params".
>
>     - Note that historically ATF uses dotted style of metadata naming, while
>       Kyua uses underscore style. Hence "execenv.jail.params" vs.
>       "execenv_jail_params".
>
>     3 kyua.conf, kyua CLI
>
>     - The new "execenvs" engine configuration variable can be set to a list
>       of execution environments to run only tests designed for. Tests of not
>       listed environments are skipped.
>
>     - By default, this variable lists all execution environments supported
>       by a Kyua binary, e.g. execenvs="host jail".
>
>     - This variable can be changed via "kyua.conf" or via kyua CLI's "-v"
>       parameter. For example, "kyua -v execenvs=host test" will run only
>       host-based tests and skip jail-based ones.
>
>     - Current value of this variable can be examined with "kyua config".
>
>     [markj] This feature has not landed upstream yet.
>     See the discussion in https://github.com/freebsd/kyua/pull/224 .
>     Having the ability to automatically jail tests allows many network tests
>     to run in parallel, giving a drastic speedup.  So, let's import the
>     feature and start using it in main.
>
>     Signed-off-by:  Igor Ostapenko <pm@igoro.pro>
>     Reviewed by:    markj, kp
>     Tested by:      markj, kp
>     MFC after:      3 months
>     Differential Revision:  https://reviews.freebsd.org/D45865
> ---
>  contrib/kyua/AUTHORS                              |   1 +
>  contrib/kyua/cli/cmd_config_test.cpp              |  12 +-
>  contrib/kyua/doc/kyua.conf.5.in                   |  13 +-
>  contrib/kyua/doc/kyuafile.5.in                    | 103 +++++++-
>  contrib/kyua/drivers/report_junit_test.cpp        |   8 +
>  contrib/kyua/engine/atf.cpp                       |  11 +-
>  contrib/kyua/engine/atf_list.cpp                  |   4 +
>  contrib/kyua/engine/config.cpp                    |  18 ++
>  contrib/kyua/engine/execenv/execenv.cpp           |  74 ++++++
>  contrib/kyua/engine/execenv/execenv.hpp           | 149 +++++++++++
>  contrib/kyua/engine/execenv/execenv_host.cpp      |  52 ++++
>  contrib/kyua/engine/execenv/execenv_host.hpp      |  63 +++++
>  contrib/kyua/engine/plain.cpp                     |   7 +-
>  contrib/kyua/engine/requirements.cpp              |  33 +++
>  contrib/kyua/engine/scheduler.cpp                 | 274 ++++++++++++++++++-
>  contrib/kyua/engine/scheduler.hpp                 |   1 +
>  contrib/kyua/engine/tap.cpp                       |   7 +-
>  contrib/kyua/examples/kyua.conf                   |   3 +
>  contrib/kyua/integration/cmd_config_test.sh       |   2 +
>  contrib/kyua/integration/cmd_report_junit_test.sh |   8 +
>  contrib/kyua/integration/cmd_report_test.sh       |   2 +
>  contrib/kyua/main.cpp                             |   3 +
>  contrib/kyua/model/metadata.cpp                   |  76 ++++++
>  contrib/kyua/model/metadata.hpp                   |   5 +
>  contrib/kyua/model/metadata_test.cpp              |   8 +-
>  contrib/kyua/model/test_case_test.cpp             |   3 +-
>  contrib/kyua/model/test_program_test.cpp          |  13 +-
>  contrib/kyua/os/freebsd/execenv_jail.cpp          |  78 ++++++
>  contrib/kyua/os/freebsd/execenv_jail.hpp          |  65 +++++
>  contrib/kyua/os/freebsd/execenv_jail_manager.cpp  |  63 +++++
>  contrib/kyua/os/freebsd/execenv_jail_manager.hpp  |  54 ++++
>  contrib/kyua/os/freebsd/execenv_jail_stub.cpp     |  75 ++++++
>  contrib/kyua/os/freebsd/main.cpp                  |  54 ++++
>  contrib/kyua/os/freebsd/main.hpp                  |  41 +++
>  contrib/kyua/os/freebsd/utils/jail.cpp            | 306 ++++++++++++++++++++++
>  contrib/kyua/os/freebsd/utils/jail.hpp            |  64 +++++
>  contrib/kyua/utils/config/nodes.ipp               |   9 +-
>  contrib/kyua/utils/process/executor.cpp           |  42 +++
>  contrib/kyua/utils/process/executor.hpp           |   1 +
>  usr.bin/kyua/Makefile                             |  15 +-
>  40 files changed, 1792 insertions(+), 28 deletions(-)
>

This broke the gcc build:
https://ci.freebsd.org/job/FreeBSD-main-amd64-gcc13_build/1307/

With several instances of
error: 'noreturn' function does return [-Werror]

From my meta files:
gcc13/usr/src/freebsd/amd64.amd64/usr.bin/kyua/engine_atf.o.meta:/usr/src/freebsd/contrib/kyua/engine/atf.cpp:199:1:
error: 'noreturn' function does return [-Werror]
gcc13/usr/src/freebsd/amd64.amd64/usr.bin/kyua/engine_atf.o.meta:/usr/src/freebsd/contrib/kyua/engine/atf.cpp:230:1:
error: 'noreturn' function does return [-Werror]
gcc13/usr/src/freebsd/amd64.amd64/usr.bin/kyua/engine_plain.o.meta:/usr/src/freebsd/contrib/kyua/engine/plain.cpp:113:1:
error: 'noreturn' function does return [-Werror]
gcc13/usr/src/freebsd/amd64.amd64/usr.bin/kyua/engine_tap.o.meta:/usr/src/freebsd/contrib/kyua/engine/tap.cpp:160:1:
error: 'noreturn' function does return [-Werror]

I'm unsure if [[noreturn]] on a virtual method is defined in the way
that this code apparently expects.

Ryan