ATF Test Cases

[Garrett Cooper]

Hi again Peter!

On Wed, Mar 26, 2014 at 3:26 PM, Kilner, Peter <Peter.Kilner at emc.com> wrote:
> Hello,
>
> I hope I am posting this question to the correct list.
>
> Is there a way to have a single ATF test case report multiple pass/fail results?  Currently I have seen that one can include a number of test conditions in a single test case (for example many aft_checks).  However the test case will only report one pass/fail result for that case.
>
> I would like to build a test case that will iterate though many configurations and would like to see a pass/fail for each config.  However this is difficult to implement with multiply test cases because of the for loops that I am using.
>
> Any suggestions would be appreciated.

As others have suggested, your approach could be tweaked a bit to
iterate over all of the different configurations you are trying to
test. In general I would create either a generic library or test
driver with common code and have a common validation method, then
build off that with additional validation methods if needed. I do that
all the time.

I think that Julio's suggestion with dynamically eval'ing tests is
spot on as well. There's an example here of how to do that (that we
might want to extend to a more generic form and push into Kyua or
FreeBSD's test infrastructure if it's ok):
https://github.com/yaneurabeya/freebsd/blob/master/bin/sh/tests/functional_test.sh
. Important: this code is proof of concept -- please do _NOT_
copy-paste it if at all possible.

Thanks,
-Garrett