svn commit: r277876 - in projects/zfsd/head/tests/sys/cddl/zfs/tests: cli_root/zpool_upgrade zil

Will Andrews will at
Thu Jan 29 19:41:21 UTC 2015

On Thu, Jan 29, 2015 at 11:14:41AM -0800, NGie Cooper wrote:
> I agree in concept. The only way to really do this in ATF is a two-step:
>     atf_expect_fail "This test panics my box"
>     atf_fail "Panic!"
> This will print out a message like "expected failure: This test panics
> my box: Panic!" though in the results, which is incredibly
> confusing... That's why Julio, NetBSD, me, and others prefer:
>     atf_skip "This test panics my box"
> A "filtering" mechanism for dealing with broken/failing tests on
> FreeBSD hasn't been developed (and it kinds of invites maintenance
> pain as it numbs people to issues in tests). So while the atf_fail is
> known and expected for SpectraLogic, it's not for the Jenkins folks,
> Isilon, Microsoft, Netflix, Panzura, etc.

I suspect that marking the tests skipped means they will be ignored more
so than marking them as expected failures.  It is inefficient to manually
filter a (potentially very long) skipped list to determine which ones are
actually failures and which ones are simply because the environment isn't
set up for them or because they require functions not yet ported to FreeBSD.

In the ZFS test suite, ~150 are skipped out of ~800 total.  So, how do you
(quickly) tell which ones are actually failures?

> Bugging the symptoms sooner rather than later is better so others know
> what commands to avoid entering in (or situations to avoid in general)
> when dealing with code. Please, please bug them so others don't
> accidentally trip themselves up on the issues you've seen.

I understand that, but first I'm trying to get the test suite to run.  As I
said in reply to Steven, all of these failures will be investigated once I
have an easily repeatable mechanism, and before merging to head.

> PS Thank you again for the work that you, Alan, and the other folks at
> SpectraLogic have done! It'll be nice to add some intelligence to
> FreeBSD/ZFS with zfsd :).


zfsd won't be included in the initial test suite import, unfortunately.
There are unfinished KPI changes that zfsd depends on in order to do its
job.  However, the vast majority of the ZFS test suite does not depend on
zfsd, and we've seen many regressions over the last few years that would
have been caught if this suite were being run.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <>

More information about the freebsd-testing mailing list