Testing with lua/atf-lua reviews

Kyle Evans kevans at freebsd.org
Thu Oct 29 01:03:12 UTC 2020

On Wed, Oct 28, 2020 at 7:43 PM Kyle Evans <kevans at freebsd.org> wrote:
> On Wed, Oct 28, 2020 at 7:23 PM Enji Cooper <yaneurabeya at gmail.com> wrote:
> >
> >
> > On Oct 24, 2020, at 9:09 AM, Kyle Evans <kevans at FreeBSD.org> wrote:
> >
> > Hello!
> >
> > I've just put up for review some work I've done to allow us to write
> > tests in lua, primarily intended to test the lua libs we're writing.
> > Please feel free to add yourself or drop in for some commentary:
> >
> > - https://reviews.freebsd.org/D26928 - atf-lua(1)/atf-lua(3) itself
> > - https://reviews.freebsd.org/D26929 - Build glue for atf-lua
> > - https://reviews.freebsd.org/D26930 - atf.tests.mk infrastructure for
> > adding tests
> > - https://reviews.freebsd.org/D26931 - Build glue for atf-lua tests
> > - https://reviews.freebsd.org/D26932 - jail(3lua) tests, as a sample
> >
> > Note that D26932 has an additional hard dependency on the libjail
> > bindings and some additions I've made to them, notably: D26080,
> > D26756, and D26927.
> >
> >
> > Hi Kyle,
> >
> > I realize that I haven’t been fully in the loop lately due to time and focusing on other things, but I’m not fully onboard with this approach.
> >
> > In particular, one of the things that jmmv was more onboard with was limiting atf, not extending it, and I agree with his desire to not do that.
> >
> Sure-
> > Furthermore, why isn’t this using the luaunit framework instead and the support being added to kyua to support luaunit:  http://lua-users.org/wiki/UnitTesting ? There are a ton of caveats with ATF that I would rather not support longer than necessary and having to teach folks how to use a homegrown test infrastructure instead of leveraging an open source test infrastructure which is supported by an external group. Doing the latter makes maintenance easy for us and improves the utility of the support better.
> >
> I had actually considered this, but ruled it out due to a couple
> factors. The main one was that there's really no benefit with adopting
> yet another test framework for base -- I think we're much less likely
> to get people that are already familiar with luaunit contributing to
> our base tests than we are to get people in one of two other camps:
> 1. They already work with our vast array of other ATF !lua tests and
> would find themselves staring at a really familiar interface, or
> 2. They don't, they want to work on Lua stuff and Lua tests in base,
> then an ATF lua interface will be at least somewhat applicable to
> other areas of our test infrastructure
> I haven't dumped all that much time into this, though, so I have no
> problem taking another aerial glance at it.

Oh, oh dear. luaunit already supports TAP output:

I kinda prefer the minimal verbosity text format that it defaults to,
but as far as effort to get off the ground goes... that's likely
impossible to beat.

More information about the freebsd-testing mailing list