Rigorous specification for TCP, UDP, and Sockets

[Peter Sewell]

>> >Are you plannning to share the tools as
>> >well?  That's what I'd be most interested in seeing, basically the
>> >ability to turn your conformance tests into regression tests.
>> 
>> It's unclear at the moment - we'd certainly like to make that
>> regression testing more routine, but the current tools are
>> >non-trivial
>> to drive.  We'll be looking to see how much interest there is, and
>> also how accessible the spec is, before going further.
>
>In what way are they hard to drive?  Are they difficult to set up, or
>difficult to run?  

both, I'm afraid - there's quite a complex infrastructure, both for
generating tests and for running the checker over them (in parallel on
a server farm), then interpreting the results of the checker is
non-trivial.  One needs to iterate the test
generation/checking/spec-fixing loop many times. 

>If they are difficult to set up, you might take a
>look at using EmuLab (www.emulab.net) to build images with configured
>tests so OSes just need to be updated to test a change.

Interesting - I didn't know about that (though I don't think it helps right now).


>> What kind of regression testing is in use now?
>
>None for the most part.  People test some limited things, but I don't
>think anyone does the kind if rigorous testing we should be doing.

ah :-(

cheers,
Peter