Re: orch(1) for tty testing

On 1/16/24 13:22, Mark Johnston wrote:
> On Mon, Jan 15, 2024 at 01:54:54PM -0600, Kyle Evans wrote:
>> Hi,
>>
>> I wrote a tool for tty testing w/ lua scripts, inspired by expect(1) and
>> with some similarities, but explicitly without the goal of trying to be
>> compatible:
>>
>> https://git.kevans.dev/kevans/orch
>>
>> An example of its use is here:
>>
>> https://git.kevans.dev/kevans/tty-tests/src/branch/main/test_canon.orch
>>
>> Each spawn() creates a new pts(4) and allows us to drive I/O to the process
>> with write/match statements.  I have one or two tests written with it so far
>> and the end result is, IMO, much less painful looking and less error prone
>> than the equivalent in C.
>>
>> It still needs a bit of polish (e.g., improved diagnostics when a match
>> fails), but I'd like to consider importing this somewhere for use in the
>> near future for tty testing given how much it simplifies.  I can push it to
>> ports, but we have a private liblua in base so it could be built and
>> banished off into, e.g., /usr/tests/sys/kern/tty/, to make my life just a
>> tad easier re: getting these to run in CI.  It's pretty lightweight; the
>> orch binary on my amd64 system clocks in at ~27K, and orch.lua at 13K.
>>
>> Any objections to importing this new tool into /usr/tests or maybe even
>> /usr/libexec?
> 
> This looks quite handy.  I have some lua scripts which I use to checkout
> a src tree, build a VM image, boot the image and run tests, then
> download the resulting kyua DB, all with a single CLI command.  I
> currently run the hypervisor under expect(1) and send commands over a
> pipe, but that was an expedient hack and I'd rather use something
> implemented natively.  There are some existing lua expect ports, but
> orch looks simpler and I'll try it out to see if it's missing anything
> for my use-case.  More generally, we lack a test framework which lets
> one boot and interact with VMs, and orch might be a useful component of
> that.

That seems like a good exercise for it- feel free to get in touch. I 
wouldn't mind porting whatever you've written in expect(1) over and take 
the opportunity to fill in the feature gaps, regardless of whether you 
use the end result or not.