How do I extend Kyua requirements checking for FreeBSD?

[Garrett Cooper]

On Thu, Jun 26, 2014 at 12:54 PM, Garrett Cooper <yaneurabeya at gmail.com> wrote:
> On Thu, Jun 26, 2014 at 12:39 PM, Alan Somers <asomers at freebsd.org> wrote:
>> On Thu, Jun 26, 2014 at 1:05 PM, Garrett Cooper <yaneurabeya at gmail.com> wrote:
>>> Hello Julio!
>>>     I'm looking at extending Kyua to create special FreeBSD-specific
>>> functionality, and I'm wondering how I should do it. In particular
>>> there are a couple items that could be checked as prerequisite
>>> requirements before running tests to avoid running tests:
>>>     1. Is a service running?
>>>     2. Is a driver loaded?
>>>     3. Is a driver loaded as a kernel module or is it statically
>>> compiled in to the kernel?
>>>     4. Am I running on a particular filesystem?
>>>     Etc.
>>>     If I was to add these "requirements functions" to kyua, how would
>>> I query the values in an interface agnostic way? I would usually set
>>> environment variables, but this makes things potentially messier
>>> (especially in C/C++ unless I create a standalone library to handle
>>> this)/harder to grep for than having a command which fetches this
>>> information, like atf-config does for the ATF tester interface.
>>> Thank you!
>>> -Garrett
>>
>> I did it just be creating a library of shell functions that I call
>> from test bodies.  For example, something like this
>> function require_module() {
>>    kldstat $1 || atf_skip "Module $1 is not loaded"
>> }
>
> This is sort of what I'm doing, but this only works for the ATF test
> interface :/. I used kldstat -m $1 by the way for detecting statically
> compiled kernel modules, like aio:
>
> # kldstat -q -m ada && echo ada is loaded
> ada is loaded
> # kldstat -q -m foobar && echo foobar is loaded
> #
>
> It's significantly trickier with drivers that don't have moduledata_t
> which matches the kernel module that's loaded, e.g. if_em:
>
> # kldstat -m pci/em
> Id  Refs Name
> 117    1 pci/em
> # ifconfig em0
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>         options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
>         ether 00:0c:29:a3:e9:9c
>         inet 10.7.216.233 netmask 0xfffffe00 broadcast 10.7.217.255
>         nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>         media: Ethernet autoselect (1000baseT <full-duplex>)
>         status: active
> # kldload if_em
> kldload: can't load if_em: module already loaded or in kernel
>
>> But that's not useful for C and C++ tests.  And it would be way cooler
>> if it were more tightly integrated into Kyua.  Would it be possible to
>> load site-specific Lua code that would handle this sort of
>> functionality, so the test could simply say "atf_set require.modules
>> aio"?
>
> Exactly!
>
> I'm going to write the C/shell libraries for ATF at least, add some
> testcases, then we can look at making them generic via Kyuafile/the
> testcase header/etc.

I'm started work on it on github:
https://github.com/yaneurabeya/freebsd_atf_kyua_test_extensions . Any
review would be much appreciated!

Testcases coming soon of course.

Thanks!
-Garrett