ATF Test Cases

[Alan Somers]

On Wed, Mar 26, 2014 at 4:26 PM, Kilner, Peter <Peter.Kilner at emc.com> wrote:
> Hello,
>
> I hope I am posting this question to the correct list.
>
> Is there a way to have a single ATF test case report multiple pass/fail results?  Currently I have seen that one can include a number of test conditions in a single test case (for example many aft_checks).  However the test case will only report one pass/fail result for that case.

Nope.

>
> I would like to build a test case that will iterate though many configurations and would like to see a pass/fail for each config.  However this is difficult to implement with multiply test cases because of the for loops that I am using.
>
> Any suggestions would be appreciated.

Is this in C/C++ or sh?  With atf-c-api(3), you can call atf_expect()
instead of atf_require().  That way the test will continue even after
one of the expectations failed.  If multiple expectations fail, then I
think kyua will print an error message for each failed expectation.  I
could be wrong however.  Also, I don't think that equivalent
functionality exists in atf-sh-api(3).

Apart from that, you pretty much have to write multiple test cases.
Note that separate test cases run in separate processes.  If your for
loop is very long and this is a C test program, using multiple test
cases will be much slower than using a single test case with multiple
expectations.

-Alan

>
> Thank you,
>
> Peter Kilner
> Software Engineer
> EMC, Isilon Seattle
> peter.kilner at isilon.com<mailto:peter.kilner at isilon.com>
> 206-777-7859
>
>
>
> _______________________________________________
> freebsd-testing at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-testing
> To unsubscribe, send any mail to "freebsd-testing-unsubscribe at freebsd.org"