Re: Add jail execution environment support to the FreeBSD test suite

On Friday, February 23rd, 2024 at 6:30 PM, Brooks Davis <brooks@freebsd.org> wrote:
> 
> > 2 The Idea
> > 
> > The idea is not new. A test could be running in a jail -- it provides the
> > required isolation with minimum or zero effort from a test.
> 
> 
> This generally sounds good. One minor concern I have is how this would
> interact with the ability to run the test suite in a jail. This is
> imperfectly supported today (IIRC ~350 failures on amd64), but it's
> quite userful for testing sweeping userspace-only tests like libsys and
> I'd love to see support expanded and improved (failures fixed or tests
> skipped, poudriere jail support, etc).
> 

Thanks for your consideration.

As I understand it's a question to the specific tests, which needs revising
and tuning for a non-prison0 case. I saw such issues with existing tests,
frequently it was about jail restrictions. And by design, a root within a jail
cannot alter restrictions of its parent jail, i.e. it's up to the creator of
such jail. In contrast, execenv=jail asks Kyua to create a temporary jail for
a test execution, and Kyua can be asked to configure it for the needs of a
test via execenv_jail. But when Kyua itself is running within an existing jail
then expectations of the test suite should be lifted a level above,
especially for usual execenv=host based tests. Thus, it's out of Kyua's
control to help with that. And this proposal does not change or improve this
case.

Just a quick idea here. Probably, the test suite could help a creator of such
jail above Kyua itself with its metadata. For example, a test is not intended
to be run within a jail and it works fine if Kyua runs within prison0, but if
Kyua itself runs in a jail it starts failing due to missing, for instance,
allow.mlock allowence, see jail(8). A test could define
execenv_jail="allow.mlock" metadata without asking for execenv=jail, i.e. it's
like a hint for a non-prison0 case. And Kyua could aggregate and provide such
information to ease creation of a jail above the Kyua itself.

On the other end of the spectrum, we could think of a new feature of jail(8)
to be able to ask it for a jail with maximum possible set of allowed things.
Just thinking out loud.

I hope I've got the topic right.


Best regards, Igor.