Re: How to run tests without installing?

On Mon, Apr 29, 2024 at 4:03 PM Brooks Davis <brooks@freebsd.org> wrote:

> On Mon, Apr 29, 2024 at 12:34:51PM +0200, Christian Weisgerber wrote:
> > How can I run the regression tests on my work-in-progress without
> > installing it first?
> >
> > Say I'm changing something in sh.  Can I run the tests on the
> > compiled sh in /usr/obj, without having to install my potentially
> > broken work into the system?  Running "make tests" in src/bin/sh
> > doesn't seem to actually test anything.
>
> Generally speaking you can't.  This is one of the problematic things
> about the current test framework.  The best you can do for something
> like sh where you really don't want to install a broken one is probably
> installing in a jail and running tests there.
>

For library tests, though, you can run the individual test just like kyua
would, but all bets are off if it relies on anything other than the code in
the binary (and associated shared libraries). Even then, it's a royal
pain.

I usually stop one step short of a jail when I've needed to do this:
I just make installworld DESTDIR=$HOME/mumble and then do
sudo chroot $HOME/mumble. Though it doesn't work at all well
for some tests (they hate being run in a chroot), but they tend to
be networking tests anyway... But the same tests that don't work
well in a chroot work even less well in a jail. There's a crazy lot
of them that require different /dev/ entries, which is why I don't do
the chroot thing that often...

tl;dr: It's easier not to test, and that's actually a problem.

Warner