Python unittest backend for Kyua

[Alan Somers]

On Mon, Mar 2, 2015 at 4:42 PM, Julio Merino <jmmv at meroh.net> wrote:
>
>> On Feb 15, 2015, at 00:16, Alan Somers <asomers at freebsd.org> wrote:
>>
>> Not much further than when I made that post, and no.  In fact, I
>> hardly use Python at all.  Professionally, my company is all about
>> Ruby, and I would be very interested in a Ruby test/unit tester for
>> Kyua.  However, Ruby's test/unit's implementation is much messier than
>> Python's unittest, so I thought that writing a unittest tester would
>> be good practice for writing a test/unit tester.  Indeed it was.  I
>> found unittest very easy to interface to; it hardly took me any time
>> to write that tester.  But in the end I stopped working on it because
>> jmmv and I disagreed about test isolation.  Jmmv's primary interest is
>> in writing system-level tests.  He basically sees a unittest tester as
>> an alternative to atf-sh.  But my primary interest is in tying
>> together separate components' test suites and get a consistent view of
>> all results. So naturally jmmv wants the same level of isolation as
>> atf-sh provides, but I want the same level of isolation as unittest
>> provides.  In fact, atf-sh style isolation is bad for my use case,
>> because it can cause tests that were originally written for unittest
>> to fail.  It also drastically increases runtime because the Python
>> interpreter must be restarted between each test.
>
> As a matter of fact, I have changed my thoughts on this.  I would like Kyua to better support "unittest"-style test programs because that's what pretty much all testing libraries implement and what other people know... and well, this model is just much faster and equally useful for the vast majority of the cases.


Well, maybe I should dust off my old branch and get back to work.  But
I don't know how my python tester would work with your executor
branch.  If I understand correctly, the executor branch dispenses with
the *-tester binaries and calls the test programs directly from the
kyua executable, correct?  The python tester relied on the tester
being a separate binary.  The tester was actually implemented in
Python, which made it very easy for the tester to interact with
individual test cases.  How would I accomplish that on the executor
branch?

-Alan