RFC: Add required_klds metadata to Kyua

Hi FreeBSD developers,

During Kyua's execenv=jail feature implementation there were discussions to
get some help from Kyua side with kernel modules required by specific tests.
The thing got "required_klds" codename.

The problem to solve here is a well-known one: typically we invoke "kyua
test" and get a lot of "Skipped" due to many kernel module requirements. And
the pain is to know which ones need loading, especially if we target a
sub-tree of /usr/tests. Also, the FreeBSD CI needs periodic sync if a new
test is added with new module requirements. And other related issues.

I would like to share the existing ideas and vision before the
implementation itself. I propose to split it onto several subsequent
projects to ease reasoning, review, etc.


Project A: declaration & skipping
---------------------------------

Add a new metadata property to Kyua which is used by a test to define its
requirements on specific kernel modules expected being loaded. The following
interface is expected:

- Define it on ATF level, e.g. for atf-sh:

     atf_set require.klds pf pflog

- Define it on Kyuafile level:

     test_program{name="testbinary1", required_klds="pf pflog"}

- Get it listed per test case:

     > kyua list -v testbinary1 | grep required_klds
     required_klds = pf pflog if_epair

Having it declared we could simplify tests and avoid manual checks for
modules with the respective "skip test" lib calls. Thus, this new metadata
would work the same way as the existing required_programs and required_files
ones -- if some of the required modules are not loaded Kyua will skip such
test with a respective message including names of the missing modules.

An extra benefit is that we do not need to iterate it like: 1) kyua test, 2)
kldload a-missing-module, 3) repeat. Such iteration is usually needed due to
our tests typically do kldstat sequentially with "skip fast" logic. Skipping
reason information still is not aggregated to make it easy to run a single
kldload for all missing modules, but that's another story which is covered
by the following Project B.

Q1: I have doubts regarding the name of this new metadata. There are several
other options in my mind, it would be appreciated to hear opinions regarding
this:
   a) required_klds -- the original idea based on the vibe that FreeBSD Kyua
      fork is mainly for FreeBSD

   b) required_mods -- to keep it OS agnostic while Kyua still is treated
      as an OS agnostic tool, and freebsd/kyua GitHub repo actually is the
      main Kyua repo in the world today

   c) required_kmods -- a little improvement of the previous option to make
      sure it talks about "kernel modules" and leave space for other meanings
      in the future, just in case

   d) required_klds | required_kmods -- to have some official aliases
      supported, but it's expected to bring confusion instead

   e) your variant


Project B: automatic kldload
----------------------------

Having Project A implemented and the existing tests augmented with the
respective metadata we could ask Kyua to automatically load the required
kernel modules.

 From the implementation perspective, Kyua is expected to have a generic
concept of a requirement resolver. A resolver may read all metadata and
decide on its actions. If we decide on "klds" name in Project A then the
very first resolver implemented would look for required_klds metadata, if
it's present, then the mentioned kernel modules will be loaded. Such
resolver could be named "klds", and we can think of future additions like
"pkgs" resolver and so on.

Currently, I see it as a new kyua command to make it separate from the
testing process itself. The interface could be as follows:

- List all available requirement resolvers (name and short description):

     > kyua rr list

- Run all resolvers or only specified ones. It would deal with the current
   dir, and later we can think of "-k" option like "kyua test" command has.

     > kyua rr run [resolver1-name resolver2-name ...]

- It's proposed to be a sub-set of commands in case if in the future we want
   additions, e.g. dry-run or verify command without actual resolving.

Having this interface the FreeBSD CI could have a separate step to run "kyua
rr run" before the existing step which runs "kyua test". But from
development perspective it may quickly end up with a need to have a single
command for both. The following interface addition could be there:

- Run all or specified requirement resolvers before the actual testing:

     > kyua test --rr ["resolver1-name resolver2-name ..."]

Another option is to use the existing Kyua mechanisms and add it as a
configuration variable, i.e. it can be configured once in the local
kyua.conf not to specify it with every test command invocation:

     # without configuration:
     > kyua -v rr="klds pkgs" test

     # with configuration:
     > echo 'rr = "klds pkgs"' >> /etc/kyua/kyua.conf
     > kyua test  # now it does resolving first


Q2: Project B has the same set of questions about naming and style. For
example, "rr" name for kyua.conf feels non-obvious and probably it could be
named like "requirement_resolvers" to keep it self-explainable.


What do you think? Any idea, comment, or suggestion are welcome.


Best regards,
igoro