Re: orch(1) for tty testing

On Mon, Jan 15, 2024 at 01:54:54PM -0600, Kyle Evans wrote:
> Hi,
> 
> I wrote a tool for tty testing w/ lua scripts, inspired by expect(1) and
> with some similarities, but explicitly without the goal of trying to be
> compatible:
> 
> https://git.kevans.dev/kevans/orch
> 
> An example of its use is here:
> 
> https://git.kevans.dev/kevans/tty-tests/src/branch/main/test_canon.orch
> 
> Each spawn() creates a new pts(4) and allows us to drive I/O to the process
> with write/match statements.  I have one or two tests written with it so far
> and the end result is, IMO, much less painful looking and less error prone
> than the equivalent in C.
> 
> It still needs a bit of polish (e.g., improved diagnostics when a match
> fails), but I'd like to consider importing this somewhere for use in the
> near future for tty testing given how much it simplifies.  I can push it to
> ports, but we have a private liblua in base so it could be built and
> banished off into, e.g., /usr/tests/sys/kern/tty/, to make my life just a
> tad easier re: getting these to run in CI.  It's pretty lightweight; the
> orch binary on my amd64 system clocks in at ~27K, and orch.lua at 13K.
> 
> Any objections to importing this new tool into /usr/tests or maybe even
> /usr/libexec?

This looks quite handy.  I have some lua scripts which I use to checkout
a src tree, build a VM image, boot the image and run tests, then
download the resulting kyua DB, all with a single CLI command.  I
currently run the hypervisor under expect(1) and send commands over a
pipe, but that was an expedient hack and I'd rather use something
implemented natively.  There are some existing lua expect ports, but
orch looks simpler and I'll try it out to see if it's missing anything
for my use-case.  More generally, we lack a test framework which lets
one boot and interact with VMs, and orch might be a useful component of
that.