ATF Test Cases

[Kilner, Peter]

Hi Garrett,

Thanks for the example, this is great solution!  I was starting to think about creating a test case generator but this a much better way to go.

Thank you,

Peter K

On Apr 9, 2014, at 4:18 PM, Garrett Cooper <yaneurabeya at gmail.com<mailto:yaneurabeya at gmail.com>> wrote:

Hi again Peter!

On Wed, Mar 26, 2014 at 3:26 PM, Kilner, Peter <Peter.Kilner at emc.com<mailto:Peter.Kilner at emc.com>> wrote:
Hello,

I hope I am posting this question to the correct list.

Is there a way to have a single ATF test case report multiple pass/fail results?  Currently I have seen that one can include a number of test conditions in a single test case (for example many aft_checks).  However the test case will only report one pass/fail result for that case.

I would like to build a test case that will iterate though many configurations and would like to see a pass/fail for each config.  However this is difficult to implement with multiply test cases because of the for loops that I am using.

Any suggestions would be appreciated.

As others have suggested, your approach could be tweaked a bit to
iterate over all of the different configurations you are trying to
test. In general I would create either a generic library or test
driver with common code and have a common validation method, then
build off that with additional validation methods if needed. I do that
all the time.

I think that Julio's suggestion with dynamically eval'ing tests is
spot on as well. There's an example here of how to do that (that we
might want to extend to a more generic form and push into Kyua or
FreeBSD's test infrastructure if it's ok):
https://github.com/yaneurabeya/freebsd/blob/master/bin/sh/tests/functional_test.sh
. Important: this code is proof of concept -- please do _NOT_
copy-paste it if at all possible.

Thanks,
-Garrett