Please provide process for small, targeted fixes in tools/regression

[Peel, Casey]

Julio, here are 3 additional changes in tools/regression to get tests in there to run and pass via prove.

* libutil.diff - fixes test-humanize_number.c to print 1-based test numbers instead of 0-based test numbers as prove expects.
* mmap.diff - adds .t file to allow running mmap test through prove
* fifo.diff - adds a top-level Makefile to compile the code in the fifo/ subdirectories. Adds .t to run the tests via prove.

Any progress on the prior set I tossed you?

-- Casey

--
Performance Test Ninja
casey.peel at isilon.com / 206.777.7945

-----Original Message-----
From: owner-freebsd-testing at freebsd.org [mailto:owner-freebsd-testing at freebsd.org] On Behalf Of Peel, Casey
Sent: Friday, April 11, 2014 7:00 PM
To: Julio Merino
Cc: freebsd-testing at freebsd.org; bdrewery at FreeBSD.org
Subject: RE: Please provide process for small, targeted fixes in tools/regression

Exellent, thanks Julio.

Attached are 3 self-contained diffs, one per directory. Descriptions of the changes:
* flock - .c changes to fix compiler errors; .t to wrap flock binary for integrating with prove
* geom_mirror - gmirror won't let you remove the last partition in a mirror which all of these testcases do and thus fail (note that I've got these testcases plus other gmirror testcases already moved over to ATF and going through internal code reviews now, so maybe we just want to discard this)
* kqueue - kqtest outputs files in TAP-consumable format, this is a .t file to run it through prove

These were an intial salvo at getting existing tests working for our internal regression. I'm sure we'll be looking at others as well. If someone can just review the changes, I can get someone internally to commit them if necssary (although these will take more time to review than commit given how small they are).

-- Casey

--
Performance Test Ninja
casey.peel at isilon.com / 206.777.7945

-----Original Message-----
From: Julio Merino [mailto:jmmv at meroh.net] On Behalf Of Julio Merino
Sent: Friday, April 11, 2014 12:02 PM
To: Peel, Casey
Cc: freebsd-testing at freebsd.org; bdrewery at FreeBSD.org
Subject: Re: Please provide process for small, targeted fixes in tools/regression

Hello Casey,

On Apr 11, 2014, at 13:56, Peel, Casey <casey.peel at isilon.com> wrote:

> I have several small, targted fixes to files in src/tools/regression/ to:
> 
> 1)      enable existing *.t tests to run that were failing

This is not "too hard" but is certainly tricky.

I did a bunch of them about a month ago but didn't write a lot of documentation on the topic.  You can find some useful notes in http://goo.gl/rWpdDR though, which includes slides and accompanying notes for the tutorial I gave at AsiaBSDCon.  Most of that ought to be converted to actual documentation of course.

The very first step, which is arguably the hardest, is to get the tests to work while running them with the prove(1) tool.  This has several implications (an important one being that any testing logic must be removed from the Makefile).

> 2)      add new *.t files for directories that enable running TAP-enabled binaries through prove

That's good as a second step.  I suspect these new *.t files will just invoke another script (the actual test), which means the .t files will be removed when the tests are hooked into the test suite.  But that's fine: it's better to get them running with prove(1) first because then the move is simple enough.

> 3)      *.c updates to remove clang compiler warnings

Yes please.

Once the tests run with prove(1) and pass, we can bundle them into the test suite.  This roughly involves moving the code to corresponding 'tests/' subdirectories, writing Makefiles and updating the mtree.  Mostly mechanical (but very annoying when the tests provide tons of data files; see usr.bin/make/tests/).

> Can someone please provide a process for getting these approved and committed in a timely manner? I would prefer to get these upstream and then pull them down if at all possible, but I simply don't have time for these to languish for weeks in a committee.
> 
> Isilon has BSD committers (eg: bdrewery) I can leverage if that would be more expedient but given these are all testing-centric it seems like getting approval from one or more people on freebsd-testing would be best.

If you can get self-contained diffs, I can try to get them in for you (but can't make any promises as my time for FreeBSD varies significantly week to week...)

For simplicity of review, I'd appreciate at least one patch for every subdirectory within tools/regression/foo/bar/ with at least one patch to get the first tests up and running first with prove(1).  More patches are better!

The move of the fixed tests to the new infrastructure (aka the boilerplate work) is something I may be able to help with.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fifo.diff
Type: application/octet-stream
Size: 795 bytes
Desc: fifo.diff
URL: <http://lists.freebsd.org/pipermail/freebsd-testing/attachments/20140430/959887dc/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: libutil.diff
Type: application/octet-stream
Size: 1504 bytes
Desc: libutil.diff
URL: <http://lists.freebsd.org/pipermail/freebsd-testing/attachments/20140430/959887dc/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mmap.diff
Type: application/octet-stream
Size: 448 bytes
Desc: mmap.diff
URL: <http://lists.freebsd.org/pipermail/freebsd-testing/attachments/20140430/959887dc/attachment-0002.obj>