Python unittest backend for Kyua
julio at meroh.net
Wed Nov 27 14:02:44 UTC 2013
On Tue, Nov 26, 2013 at 11:47 PM, Alan Somers <asomers at freebsd.org> wrote:
> I wrote a Kyua backend for Python programs that use the unittest
> module from the standard library. It's incomplete (no man page, no
> tests, no test case isolation, and no tested version of python other
> than 2.7), but I'm posting it to discuss the approach. Do you think
> it's a good start?
> Rather than use unittest's CLI, I wrote the backend to dynamically
> load the file under test and interrogate the test using unittest as a
> library. This allowed more intimate access. For example, the
> unittest CLI doesn't have a "list" command, but the backend can still
> list tests.
> Does it look good so far?
I like the idea a lot (and I had not considered it earlier). The code
is a good start, but see below.
One concern I have is regarding how you will implement isolation
features. The more I think about them, the more I think that they are
not specific to the ATF interface at all: i.e. all tests should be
executed with the same level of isolation.
Now, in your case, it seems you'd have to reimplement these features
in Python -- which means that there will be two different
implementations for the same thing and this can easily lead to
inconsistencies. (atf had 3 versions of this in the very beginning,
one for each supported language, and it wasn't pretty!) But having a
pure Python binary has its benefits, if only for simplicity.
The alternatives would be to either 1) hook the isolation C code into
a Python module and call it from your tester, or 2) write the tester
in C using all existing code (including CLI parsing for a consistent
interface, isolation features, etc.) and just call into the Python
interpreter to list tests or run one of them.
What do you think?
Julio Merino / @jmmv
More information about the freebsd-testing