Testing the build infrastructure

[Nik Clayton]

On Fri, Dec 10, 2004 at 05:58:53AM +0200, Giorgos Keramidas wrote:
> On 2004-12-07 22:18, Nik Clayton <nik at freebsd.org> wrote:
> > Guys,
> > Given the comments that have been bandied about recently relating to
> > breakage of various parts of the build infrastructure, and functionality
> > that used to work no longer working, I'd like to make some concrete
> > progress towards fixing things. [...]
> >
> > Thoughts?
> 
> That looks like a good plan :)
> 
> Any details on how the tests should be written?  Can we (re)use the
> style of the tests of src/ at all?

Ideally that's exactly what we'll do.  I'm a big fan of prove(1), which
will run tests and generate a report of which succeeded, which failed,
and so on.  The output format that tests need to generate is listed in
src/tools/regression/README.

It's easy to write tests in Perl and /bin/sh, and I'm working on a C
implementation.  I strongly suspect that most of our tests will be
written in /bin/sh -- a lot of them will be:

  * Did this file get created?

  * Does the output match the canonical output which we've stored in
    /this/file?

  * Was the exit code the one we expected?

For example, I don't (because I've not looked in to it in any detail)
understand exactly what's broken in the obj/ building case.  But with
tests that describe how things are supposed to work I don't need to.
All I need to do (prior to making a commit) is run the tests.  As long
as they all pass everything's fine.

The only extra work (after this is in place) is when we make
infrastructure changes.  There need to be corresponding tests in place.

So, when I committed the .sgml -> .xml conversion code, if I'd committed
a test for it as well, we'd know exactly when it broke, and why.

N
-- 
FreeBSD: The Power to Serve      http://www.freebsd.org/               (__)
FreeBSD Documentation Project    http://www.freebsd.org/docproj/    \\\'',)
                                                                      \/  \ ^
   --- 15B8 3FFC DDB4 34B0 AA5F  94B7 93A8 0764 2C37 E375 ---         .\._/_)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-doc/attachments/20041211/4f36d3d9/attachment.sig>