[CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

[Garrett Cooper]

Hi Simon!

On Oct 1, 2012, at 3:31 PM, Simon J. Gerraty wrote:

> Hi Garrett,
> 
>>> From: Garrett Cooper <yanegomi at gmail.com>
>>> Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple =
>> programs instead of a singular program
>>> Date: September 2, 2012 11:01:09 PM PDT
>>> To: freebsd-hackers at freebsd.org
>>> Cc: "freebsd-arch at FreeBSD.org Arch" <freebsd-arch at freebsd.org>
>>> =20
>>> Hello,
>>>   I've been a bit busy working on porting over ATF from NetBSD, and
> 
> Thanks, we've been using ATF in Junos for a while and glad to see it 
> being imported to FreeBSD.

Thanks (and I appreciate the help marcel@ has given by taking the first step in importing it).

>>> one of the pieces that's currently not available in FreeBSD that's
>>> available in NetBSD is the ability to understand and compile multiple
>>> programs. In order to do this I had to refactor bsd.prog.mk (a lot).
> 
> A change like this to bsd.prog.mk can have considerable fallout.
> Eg. any makefile that tweaks OBJS is suddenly out of luck.

Well… any Makefiles attempting to be "clever" (e.g. crunchgen) are out of luck and would need to change. Most [include] Makefiles will see virtually no change.

> Not to mention the fact that bsd.prog.mk goes from being relatively
> simple, to unspeakably hard to read, and all for rather limited return. 

I sort of tried to match bsd.prog.mk in NetBSD in the beginning, but things diverged from there… I'm sure things could be made more readable so any comments would be much appreciated :)!

> Apart from ATF, is there any huge demand for building multiple progs in
> the same directory?

Well, I noticed that there are a couple ugly messes in the gnu/... directory that attempt to work around the fact that bsd.prog.mk doesn't support more than one program by being tricky and emulating bsd.prog.mk instead of solving the generic problem (and once I got over that compatibility issue I stopped tracking this class of consumer). Most consumers don't care (they split up programs into different directories or use hardlink hacks/basename detection to differentiate one program functionally from another).

Getting back to the idea at hand, the enhancement goal was to get the testcases to live [and optionally be built/installed] with the source code to avoid bitrot; I know this isn't the current NetBSD design, but this is what was requested be done by gnn, marcel, and mdf, and I agree. It order to do this, I need to be able to build multiple programs so things are as transparent. On the flipside, bsd.prog[s].mk can live on its own, be pulled in automatically by bsd.test.mk, and that would be it. This encourages code duplication though and bugs can persist in either Makefile, when I'd rather work out all the kinks in whatever succeeds the legacy bsd.prog.mk file.

> FWIW we use progs.mk (as bsd.progs.mk) from
> ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz 
> It isn't ideal, but it certainly avoids a lot of churn and complexity
> for what is essentially a corner case.

This requires pulling bmake into FreeBSD proper in order for things to function the last I checked as bsd.prog.mk depends upon bmake directives in order to function properly 100% of the time (and there are external dependencies and assumptions one has to deal with when using bsd.prog.mk, etc from NetBSD that made that a bit of a no-go without refactoring/pulling in bits from NetBSD). I could be wrong though, so please let me know if I am :).

Ideally however, I would like doing this compared to running a custom build infrastructure, but (as you probably know) this requires rototilling the current FreeBSD build system to a large degree (definitely not impossible… just needs time and effort).

> We have an atf.test.mk which leverages that.
> It also handles automatically running the tests if building for the
> host. This allows us to fail the build if any unit-tests do not pass.

Hmm… that wasn't really the end-goal of bsd.test.mk based on my reading, but I'm sure Julio would be interested in it. I need to do adjusting in order to allow automatically executing testcases compatible to the host architecture, but that isn't hard to do (no more than  a couple hours work).

> Note, atf.test.mk is loosely derrived from NetBSD's bsd.tests.mk, but
> named for what it is (ATF specific tests) since we wanted the
> flexibility to have more than one test framework.

bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't entirely ATF centric either. I'm trying not to stray too far from NetBSD in this area because there really isn't any reason to do so. FWIW I'd like to see other test frameworks (lua, unittest//nose, etc) just snap into bsd.test.mk as easily as possible as it would make some groups lives easier, but that requires a bit more thought for another day.

Thanks for the feedback!
-Garrett