racy tests

[Ngie Cooper]

On Tue, Apr 25, 2017 at 4:02 PM, Brooks Davis <brooks at freebsd.org> wrote:
> I've been running the FreeBSD test suite for mips64 under qemu.  As a
> result, I'm seeing some tests fail due to assumptions about timing producing
> test races.  For example one of the pwait tests does this:
>
> timeout_many_body()
> {
>         sleep 1 &
>         p1=$!
>
>         sleep 5 &
>         p5=$!
>
>         sleep 10 &
>         p10=$!
>
>         atf_check \
>                 -o empty \
>                 -e empty \
>                 -s exit:124 \
>                 timeout --preserve-status 7.5 pwait -t 6 $p1 $p5 $p10
> }
>
> Under emulation, particularly if the host disks are busy, it's easily
> possible for the first sleep to exit before pwait actually runs.
> In practice, we could probably get away with cranking up the times a
> fair bit, but that would make the test slow and the race would still
> exist.
>
> Any thoughts about the right solution?  Something not time based would
> be ideal, but then it seems like we'd need a parallel process to kill
> some of the waited for victims we quickly end up with something more
> complicated than pwait that also needs testing...

(Adding bdrewery@, testing@)
I need to think about this a bit. The issue might be that we're using
the wrong timer for sleep(1)/need to account for being interrupted.

Needless to say, emulation really screws up timing assumptions because
virtual clocks don't function like hardware clocks.

Thanks,
-Ngie