Adding new "test interfaces" to work with Kyua?

Julio Merino julio at
Mon Nov 25 02:01:54 UTC 2013

On Sun, Nov 24, 2013 at 5:34 PM, Garrett Cooper <yaneurabeya at> wrote:
> On Nov 24, 2013, at 2:29 PM, Julio Merino <julio at> wrote:
>> On Sun, Nov 24, 2013 at 5:09 PM, Garrett Cooper <yaneurabeya at> wrote:
>>>        Trying to reconcile my diffs and get things sane again, I realize that due to refactoring (and especially after I installed devel/kyua) that no longer functions for me off my branch. Are there directions for how I can program in a new test interface so I can integrate prove testing into Kyua?
>> What kind of interface do prove-based test programs have?  Are they
>> binaries/scripts that you can just run and assume that a 0 exit code
>> means passed and 1 means failed?  If yes, you'd just funnel them
>> through the 'plain' interface and they should just work as a first
>> cut.  (I believe this approach works for most test programs out
>> there.)
>> But there are some details here:
>> and more in the kyua-tester manual page and its references.
> It’s a bit more complex than that. nose and prove both implement collection and execution test features, meaning that it goes and figures out what to execute (for nose it’s a variety of parameters — filename, mode, etc — that determine whether or not it’s testable; for prove it’s all files with a .t extension), then executes it.

I need to read about that a bit more then. Is there a web site for
these tools? Their names are so generic that it's hard to find stuff.

That said, I have started to take a look at the existing .t tests...
and the majority of them seem to be just wrappers that execute a shell
script.  In these cases, I think we should just drop the .t and use
the plain interface to execute the existing shell script.  By pure
coincidence, I had been working today on "porting" a couple of these
tests; expect a patch for the proof of concept soon.

> I’d like to take the error messages and make them more meaningful though, because it doesn’t help if something fails and I need to diagnose why it failed, but lost the info I need to debug the problem...

As long as these print the error messages to either stdout or stderr,
they will be captured by Kyua and stored into the reports.

Julio Merino / @jmmv

More information about the freebsd-testing mailing list