100% repeatability?

[Julio Merino]

[ Thread hijack coming! ]

On Thu, Feb 27, 2014 at 12:20 PM, Alan Somers <asomers at freebsd.org> wrote:
>
> If nprocs and maxproc were jail-specific variables, then you could
> simply run this test inside of a jail.  However, they look global to
> me.  I think that the only good way to test the feature would be to
> run the test inside of a private bhyve instance with an extremely
> minimal set of services.  That way you could ensure that no unexpected
> processes would be started.  It would be much more work, though.  You
> would also need a way to communicate to the guest OS.  Perhaps an
> emulated serial port?

Wow... that sounds like a lot of overhead for an apparently-simple kernel test!

I think we should discuss, and possibly document, a recommended way of
running kernel-level tests for FreeBSD.  We have nothing in this
regard and reinventing the wheel for every test without thinking
through the various alternatives upfront will be... "ugly" to say the
least.

Now, I do not have an answer to this.

>From previous experience in NetBSD, I think that the "anykernel"
design implemented there works very well for these kinds of tests.  I
do think FreeBSD would benefit from rump immensely, but the problem is
that it's a complex thing to do implementation-wise. See
http://en.wikipedia.org/wiki/Anykernel and
http://wiki.netbsd.org/rumpkernel/ to get started.

Without rump, all that is left are jails, bhyve instances and
natively-run tests (and maybe something else?).  Each has its own
problems but are easier to use today than reimplementing rump.  Maybe
we should attempt to guide developers to one of these three options
depending on what they are trying to test and provide helper code to
make the setup easy?  Or maybe we should investigate whether "porting"
rump to FreeBSD would make sense?