FCP 20190401-ci_policy: CI policy

Ed Maste emaste at freebsd.org
Thu Aug 29 23:36:17 UTC 2019


On Thu, 29 Aug 2019 at 07:41, Konstantin Belousov <kostikbel at gmail.com> wrote:
>
> More, I know that tests are of very low quality, which means that
> brokeness of the tests is not an indicator of anything until root cause
> is identified.

"Low quality" needs clarification here. I can think of many attributes
of a test that might lead someone to claim tests are low quality:

- The test result is not consistent (e.g., a "flaky test")
- The test does not actually test what it claims to
- The test does as it claims, but there is no value in the result
- Test coverage overall is insufficient (i.e., not an issue with a
specific test)
- The test has excessive requirements (run time, memory usage, etc.)
- The test is difficult to maintain


More information about the freebsd-hackers mailing list