Fwd: Getting rid of the ATF tools

[Garrett Cooper]

(Pardon my message mangling as I'm forwarding this to freebsd-testing@
for context)

Hello!

This is a follow-up on our previous conversation to explicitly discuss
the removal of the ATF tools from the src tree.

(I'll be happy to send this to any public lists you think are
appropriate, but first wanted to get a sense of the feasibility by
running it by you.  Or, rather, should I file a PR with the proposal
and patches?)

The reasons behind this proposed change are:

- The ATF tools (atf-run, atf-report mainly, but also atf-config and
  atf-version) are deprecated.  They do not and will not receive new
  features.  atf-run also has some nasty race-condition bugs that are
  very rare, but do trigger periodically.

- Keeping the ATF tools around hinders the development of Kyua (the name
  of the project to replace the ATF _tools_ only) because, as long as
  the ATF tools are around and the tree needs to remain compatible with
  them, one cannot take advantage of the new features in Kyua.

- One of the new features in Kyua is the ability to plug test programs
  of various interfaces into the same test suite.  At the moment, this
  means that you can mix both ATF-based test programs and "standard"
  test programs transparently into a single test suite definition.
  Dropping the ATF tools allows the FreeBSD src tree to define such
  more-advanced test suites (because one needs not be restricted by the
  limitations of the Atffiles), with the benefit that existing test
  programs can be run along ATF test programs in one go and get the
  results in one single report.

- Because ATF is not yet really used within FreeBSD and because users
  are not yet familiar with the ATF tools, these can be dropped today
  without any impact.  There is no need to later implement complex
  migration procedures nor retrain users if FreeBSD starts using the new
  tools from the beginning.

- And, to a lesser extent, shape the development of Kyua.  If FreeBSD
  starts using Kyua directly, I'm sure there will be several feature
  requests against the project and those will shape how new releases
  look like.

Notes:

- This does NOT affect the ATF libraries.  atf-c, atf-c++ and atf-sh,
  which are quite small overall, remain in the tree.

- This does NOT propose the import of Kyua into the src tree.  Kyua will
  remain in ports for the time being.  This is not an issue because Kyua
  is only the run-time engine for the tests and is not needed to build
  the tests that may be added to src.

- Of course, *.test.mk should be changed to detect whether Kyua is
  installed or not and act appropriately when it is.  E.g. when doing
  'make test', error out if Kyua is missing and point the user at how to
  install it, and otherwise use the binary from ports.

The necessary patches to do this are attached.  Sorry Garret; these
remove quite a bit of things that you did... :-P

Thoughts?

Thanks!

gcooper> A couple things:
gcooper> 1. I don't mind if you unwind my work a bit. It is what it is
and I'm not really invested in it anymore other than it helps me get
my job done today. I've lost interest in pushing this into FreeBSD
because I have better things to do than fight bikeshedders and wait
for hell to freeze over.
gcooper> 2. Please move the OptionalObsoleteFiles.inc entries
[re]moved to ObsoleteFiles.inc as they need to be removed from systems
running CURRENT that have installed atf in the past (like my VM for
instance). There aren't many people using ATF I suspect, but for those
of us that are, it would be nice if the files were yanked completely.
gcooper> 3. Please put in the needed bits for generating Kyuafiles in
atf.test.mk.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: remove-atf-tools.diff
Type: text/x-diff
Size: 33533 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-testing/attachments/20130719/c6556efb/attachment.diff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rename-tests-user.diff
Type: text/x-diff
Size: 1049 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-testing/attachments/20130719/c6556efb/attachment-0001.diff>