Adding new "test interfaces" to work with Kyua?
Julio Merino
julio at meroh.net
Mon Nov 25 02:01:54 UTC 2013
On Sun, Nov 24, 2013 at 5:34 PM, Garrett Cooper <yaneurabeya at gmail.com> wrote:
> On Nov 24, 2013, at 2:29 PM, Julio Merino <julio at meroh.net> wrote:
>
>> On Sun, Nov 24, 2013 at 5:09 PM, Garrett Cooper <yaneurabeya at gmail.com> wrote:
>>> Trying to reconcile my diffs and get things sane again, I realize that due to refactoring (and especially after I installed devel/kyua) that prove.test.mk no longer functions for me off my branch. Are there directions for how I can program in a new test interface so I can integrate prove testing into Kyua?
>>
>> What kind of interface do prove-based test programs have? Are they
>> binaries/scripts that you can just run and assume that a 0 exit code
>> means passed and 1 means failed? If yes, you'd just funnel them
>> through the 'plain' interface and they should just work as a first
>> cut. (I believe this approach works for most test programs out
>> there.)
>>
>> But there are some details here:
>>
>> https://code.google.com/p/kyua/wiki/TestersDesign
>>
>> and more in the kyua-tester manual page and its references.
>
> It’s a bit more complex than that. nose and prove both implement collection and execution test features, meaning that it goes and figures out what to execute (for nose it’s a variety of parameters — filename, mode, etc — that determine whether or not it’s testable; for prove it’s all files with a .t extension), then executes it.
I need to read about that a bit more then. Is there a web site for
these tools? Their names are so generic that it's hard to find stuff.
That said, I have started to take a look at the existing .t tests...
and the majority of them seem to be just wrappers that execute a shell
script. In these cases, I think we should just drop the .t and use
the plain interface to execute the existing shell script. By pure
coincidence, I had been working today on "porting" a couple of these
tests; expect a patch for the proof of concept soon.
> I’d like to take the error messages and make them more meaningful though, because it doesn’t help if something fails and I need to diagnose why it failed, but lost the info I need to debug the problem...
As long as these print the error messages to either stdout or stderr,
they will be captured by Kyua and stored into the reports.
--
Julio Merino / @jmmv
More information about the freebsd-testing
mailing list