Plugging ATF tests into the build and other cleanups

[Garrett Cooper]

On Oct 28, 2013, at 8:47 AM, Alan Somers <asomers at freebsd.org> wrote:

> On Sun, Oct 27, 2013 at 7:31 PM, Julio Merino <julio at meroh.net> wrote:
>> On Sun, Oct 27, 2013 at 8:25 PM, Eitan Adler <lists at eitanadler.com> wrote:
>>> On Sun, Oct 27, 2013 at 6:12 PM, Julio Merino <julio at meroh.net> wrote:
>>>> Hello!
>>>> The one concern I have here is having to keep track of all tests in
>>>> tools/build/mk/OptionalObsoleteFiles.inc so that setting
>>>> WITHOUT_TESTS=no cleans up /usr/tests. This will be a pain to maintain
>>>> and a sure source of inconsistencies. If we could special-case this to
>>>> make it more automatic, do you have any suggestions?
>>> 
>>> Is it possible to use the list of current tests and just delete any
>>> files which are not listed?
>> 
>> I think what you are suggesting applies to src/ObsoleteFiles.inc, not
>> tools/build/mk/OptionalObsoleteFiles.inc?
>> 
>> For deleted tests, I think using src/ObsoleteFiles.inc is fine as
>> usual. Deleted tests have to be removed no matter what the value of
>> MK_TESTS is. If that list gets out of hand at some point we could
>> revisit this, although I'm not sure how you can easily determine the
>> list of "current tests". AFAIK there is no list in src detailing all
>> files that are expected to be installed?
>> 
>> My concern is only about the latter at the moment. When MK_TESTS=no,
>> /usr/tests should not exist at all and, therefore, a "make
>> delete-old-files" should wipe it. We can do this as usual, with the
>> functionality in tools/build/mk/OptionalObsoleteFiles.inc to record
>> all files to be deleted, or we can do something different to avoid
>> maintaining the list by hand. A simple "rm -rf ${DESTDIR}/usr/tests"
>> would suffice, but I'm just wondering if that'd be an acceptable thing
>> to do.
>> 
>>> We would need some way to have "local" tests but that should not be difficult?
>> 
>> You mean tests from ports and/or for manually installed software?
>> Those should all be somewhere in /usr/local/ and we can easily hook
>> them into the test suite.  (Hadn't thought about these, but it's
>> easy.)
> 
> On our custom version of FreeBSD, we have a few ports that install
> tests to /usr/local/tests, a top-level Kyuafile at both
> /usr/tests/Kyuafile and /usr/local/tests/Kyuafile, and a symlink from
> /usr/tests/local -> /usr/local/tests.  It works well.

It would be nice if this was standardized and documented like with gcc for instance with its debug lookup paths…
Thanks!
-Garrett