Non-stable tests

[Olivier Cochard-Labbé]

Hi,
On our CI (Jenkins running the full kyua test suite on a bhyve VM), some
tests are sometimes failing without a valid cause.
So I had to run a second time the kyua test suite to confirm it's a "real"
regression, and the second time I didn't meet any problem.
The last 2 tests that meet this problem are:
- sbin.devd.client_test.seqpacket, failed with "Test case body timed out"
- lib.libc.sys.getitimer_test.setitimer_old, failed with "setitimer(2) did
not return remaining time, failed with"

I've tried to rerun these tests a thousand times, but I didn't reach to
reproduce the problem: they've succeeded each time after.

So I wonder if there is a problem about "non stable" tests ?

I've done a rapid search on ci.freebsd.org and I've found an example.
In this series:
https://ci.freebsd.org/job/FreeBSD-head-amd64-test/10391/ (r344839)
https://ci.freebsd.org/job/FreeBSD-head-amd64-test/10392/ (r344840)
https://ci.freebsd.org/job/FreeBSD-head-amd64-test/10393/ (r344843)

=> with job 10392, the test "lib.libarchive.functional_test.test_fuzz_zip"
failed, but there is no change in the code regarding libarchive during
these 3 commits revision period (
https://svnweb.freebsd.org/base/head/?view=log&pathrev=344843).
So why it failed once here ?
This is the same symptom I meet on my platform too.

Regards,

Olivier