Re: Add jail execution environment support to the FreeBSD test suite

On Friday, February 23rd, 2024 at 10:30 AM, Paul Floyd <paulf2718@gmail.com> wrote:
> 
> Will there be an option to turn this off globally?
> 
> I sometimes run test suits with Valgrind. I'm not aware of any option to
> do that cleanly so I just do it with brute force: run kyua and all
> children under Valgrind.
> 
> valgrind --trace-children=yes -q kyua test -k Kyuafile
> 
> Adding jails into the mix is likely to give worse results. I did try
> using an x86 jail as an alternative to VirtualBox but I gave up as I got
> too many test failures.
> 

Thanks for your attention and mention of your use case.

I've added this point to my TODO. I guess it could look as the following:

	valgrind --trace-children=yes -q kyua -v execenv=host test -k Kyuafile

It would skip tests designed for other execution environments and run only
"host" ones.

I think execenv Kyua config variable could be a list actually. By default, if
jail feature is present it could be set to "host jail", i.e. to run tests
designed for all environments.

As long as tests which opted-in to use execenv="jail" seem to be out of your
interest in your specific case it looks to be as one of the options. I was
thinking about possiblity that a test could be ready to run within "host" and
"jail" env, I actually wrote such tests and they work, but it needs more
effort from a test author and can be error prone. Anyway, Kyua itself cannot
help with duality of a test. Some tests can be freely run as "host" or "jail"
without any change, others may need quite an effort to prepare for both cases.

Does it look like something that would cover your production needs?


Best regards, Igor.