Available Tests

[Fabian Keil]

Alex Denisov <1101.debian at gmail.com> wrote:

> I just built FreeBSD 10 using “WITH_TESTS=1”, and run tests from ‘/usr/tests’ using ‘root' user.
> 
> I have found that there are ~3,5k test results, which makes me curious: are there other tests?

Yes. If you take a closer look at /usr/src you'll find various other
tests that aren't integrated into the kyua/atf tool chain (yet).

I also believe that your ~3,5k test results represent more
that ~3,5k individual tests due to suboptimal integration of
other test suites.

Various external test frameworks are also potentially relevant
for FreeBSD, but have too much dependencies to integrate.

As an example, a TCP/IP regression was found through Privoxy-Regression-Test:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=173309

Obviously the regression could have been found with a
FreeBSD-specific test as well, but somebody would have
to spent the time to write such a test first.

> I’m asking because, for instance, LLVM test suite has ~17k tests
> (unit, regression, and performance tests).

Some of these tests are probably also run on FreeBSD systems
to test the clang integration ...

> FreeBSD seems to be bigger, but has less tests.

Unlike LLVM, FreeBSD contains a lot of old code from a time
where automated testing was less common.

Comparing it with other operating systems might make more
sense.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-testing/attachments/20160401/a7605b7b/attachment.sig>