Re: Add jail execution environment support to the FreeBSD test suite

On Friday, February 23rd, 2024 at 7:57 PM, Paul Floyd <paulf2718@gmail.com> wrote:
> On 23-02-24 16:08, Igor Ostapenko wrote:
> 
> > On Friday, February 23rd, 2024 at 10:30 AM, Paul Floyd paulf2718@gmail.com wrote:
> > 
> > > Will there be an option to turn this off globally?
> > > 
> > > I sometimes run test suits with Valgrind. I'm not aware of any option to
> > > do that cleanly so I just do it with brute force: run kyua and all
> > > children under Valgrind.
> > > 
> > > valgrind --trace-children=yes -q kyua test -k Kyuafile
> > > 
> > > Adding jails into the mix is likely to give worse results. I did try
> > > using an x86 jail as an alternative to VirtualBox but I gave up as I got
> > > too many test failures.
> > 
> > Thanks for your attention and mention of your use case.
> > 
> > I've added this point to my TODO. I guess it could look as the following:
> > 
> > valgrind --trace-children=yes -q kyua -v execenv=host test -k Kyuafile
> 
> 
> Sounds good, but I'd need to test in order to make sure. I don't run
> that kind of test very often.
> 
> > Does it look like something that would cover your production needs?
> 
> 
> It's not to test FreeBSD for prod, it's to test Valgrind is working with
> all those tests (any FreeBSD errors picked up are a bonus).
> 

Paul, it would be appreciated if you have time to test the patch. The general
test plan is covered here: https://reviews.freebsd.org/D42350, but specifically
for your case the following extra check could be run in addition:

valgrind --trace-children=yes -q kyua -v execenv=host test -k /usr/tests/sys/netpfil/pf/Kyuafile

It should report that pass_block:* tests are skipped due to the given
"-v execenv=host" runtime configuration.

I hope current implementation meets your expectations.


Best regards, Igor.