[RFC] Rename `make test` in suite.test.mk with `make regress`

[Julio Merino]

> On Oct 17, 2015, at 21:56, NGie Cooper <yaneurabeya at gmail.com> wrote:
> 
>> 
>> On Oct 17, 2015, at 18:50, Julio Merino <jmmv at freebsd.org> wrote:
>> 
>> On Oct 17, 2015, at 18:03, NGie Cooper <yaneurabeya at gmail.com> wrote:
>>> 
>>> Hi all,
>>> 	There’s a lesser known target in suite.test.mk that runs `kyua test` in a similar manner to how Jenkins and other groups have integrated kyua into their test infrastructures.
>>> 	The legacy target on FreeBSD was `regress`, but the target created with the bsd.test.mk creation back a few years ago was `test`. Why change from `test` to `regress`? There are places in the tree (bin/test for example) that have targets named test, so in order to avoid clashing with a common target (name), it’s best to use the legacy target name.
>>> 	Would anyone have any serious heartburn over the change?
>> 
>> Is this only because of bin/test?  Seems like renaming a target to avoid that one collision is just moving the problem around.  It is possible that some other directory could later grow a target that conflicts with your new name.
> 
> Yes, I realize that. The goal though is I want to be able to call `make <something>` from the top and it would iterate down each and every subdirectory and run tests.

Other for the bin/test naming collision, that should work already, shouldn't it?

Not sure how ugly this would be, but you'd also define an internal "run-tests" target or similar that is used exclusively by the recursion code.  Then, in the top-level Makefile, you define a bare "test" target that starts the recursion so that "make test" works.  And then, in bsd.test.mk, you define a "test" target if - and only if - there is no "test" target yet.  bin/test would behave differently than the rest if you ever went in there and ran "make test", but I think that's a minor problem.

This is offtopic, but do you have a plan anywhere on how to make the tests invocation this way reliable?  Without installed binaries, the tests might not do the right thing, and I'm not sure if the object tree is enough for this to work properly.  That's the reason why "make test" from the top-level directory is currently disallowed.

> Sadly, most people don’t care to figure out how kyua works enough to run `kyua test`… so I’m just trying to lower the barrier of entry in a way that will work 100% of the time.
> 
>> Strictly speaking, "regress" is wrong.  We do not have regression tests only.  Also, "regress" is a pretty obscure name for a target; it does not appear in any other projects nor in any other build systems that I know of.
> 
> Unfortunately it’s been in place a lot longer than anything else, so it’s kind of the “defacto standard”. That said...

I think I see what you mean: that because "regress" already existed in the FreeBSD tree, no other targets were introduced to collide with it?  If yes, I wouldn't expect that to be the case.  Tests were not interleaved with the source tree in the past, so the "regress" target only existed in a subset of the tree, didn't it?

>> Have you considered "check"?  That'd be in line with what automake does, for example, which would homogenize the target name with a ton of other projects out there.
> 
> … `make check` would be ok too. I just have to comb the tree for binaries that don’t match `check`.
> 
> Thanks!
> -NGie