Re: git: 0849f1634a70 - main - tests/netinet: add test for IP_MULTICAST_IF

On 24 Mar 2025, at 15:12, Gleb Smirnoff wrote:
> On Mon, Mar 24, 2025 at 11:45:55AM +0900, Kristof Provost wrote:
> K> >     tests/netinet: add test for IP_MULTICAST_IF
> K> >
> K> It bugs me a little that we have to jump through hoops to get a C program
> K> running in a specific jail/network setup.
> K> It’s somewhat common for that to happen, and I’m sure there’d be more cases
> K> if it were easier to do (e.g. the getaddrinfo tests you’ve been working on
> K> could be even more useful with a custom DNS server, so in their own vnet
> K> jail).
>
> It bugs me, too. :) Igor has already pushed changes that allow you to specify
> in the Makefile:
>
> TEST_METADATA.foo+= "execenv="jail" execenv_jail_params="vnet"
>
The primary intent behind that was to allow for parallel test execution.
A lot of tests, and a lot of this is my fault, use the same network ranges and jail names, so if we run them at the same time they’d conflict and break in unexpected ways.
Running them inside a vnet jail avoids this issue.

I was actually looking at adding this to your multicast tests, but you don’t assign addresses on the host’s epair, so there’s not much point.

> Then this gets into Kyuafile after build. This is a step forward, but not
> enough. First problem is that the created jail is empty and doesn't have any
> interfaces. Second problem is that the specified environment affects all tests
> inside one test file.
>
> Ideally we'd like to have a declarative language to describe jail environment
> for a test.  This is basically what vnet.subr does for shell tests and what
> atf_python.sys.net.vnet does for python tests, but brought to kyua level and of
> course agnostic to the language of a test.  Igor is thinking in this direction
> and hopefully will eventually deliver something.
>
Interesting idea. I fear it would be difficult to get something that’s expressive enough to cover all the cases we care about, while still being simple enough to be easy to use.

On the other hand, perhaps we don’t need something that covers 100% of cases. We’re probably never going to be able to build the sorts of complex setups (for values of complex that are ‘three jails with routes, a simple daemon in one or more and some pf rules) I often build for pf tests, but perhaps we can cover 80% of cases with a simple “Build a jail with one interface with a known address and a default route”.

Perhaps a ‘execenv_jail_init=foo.sh’ calling a separate script to do the setup?

> K> The downside is that we’re compiling the C test code on every run, but I
> K> expect test programs to be tiny, so that’s not too much of a cost.
> K>
> K> Does this seem useful to you too?
>
> That would fail on decreased installation, that has tests, but doesn't have
> compiler.  This is not just a made up problem, this is what many would do in
> CI, cause you want usually as quick as possible response time from CI, and if
> your team is not hacking the compiler, you won't add compiler to the CI build.
>
That’s a good point, and not something I think we can reasonably do from atf-sh. Not unless we do a pre-processing step during ‘make install’ (on top of the ‘add the shebang line’ we already do) to extract C code, build it and install it somewhere the test can find it.

I think I don’t dislike the current approach enough to try that.

Best regards,
Kristof