svn commit: r263778 - in head: bin lib lib/clang sbin share/mk usr.bin usr.sbin

[Warner Losh]

The dependency would have to be on the artificial targets that
are created, and could easily be done. .WAIT is good for things
like libc where everything is dependent on it, but poor for other
dependencies which would be expressed as individual
dependencies.

Warner

On Apr 24, 2014, at 12:14 PM, Ian Lepore <ian at FreeBSD.org> wrote:

> I couldn't think of a way to do that, because the directory itself isn't
> the dependency.  That is, whether one directory has a newer/older
> timestamp than the other isn't a function of the build process.  The
> fact that someone else mentioned inserting .WAIT into the list is
> something I took as a confirmation that the regular dependency mechanism
> wouldn't work (so maybe I didn't think about it hard enough).
> 
> -- Ian
> 
> On Thu, 2014-04-24 at 11:17 -0600, Justin T. Gibbs wrote:
>> Why wouldn’t we fix these with explicit subdir dependencies (e.g: subdira: subdirb)?  I’m pretty sure I had this working with a hacked up bsd.subdir.mk at a previous job, but my memory is hazy.
>> 
>> —
>> Justin
>> 
>> On Apr 21, 2014, at 7:35 AM, Warner Losh <imp at bsdimp.com> wrote:
>> 
>>> (sorry for the top post)
>>> 
>>> This looks good to my eye. I’d be tempted to toss in a comment about the
>>> __wait=.WAIT construct is due to the primitive nature of bmake’s parser
>>> so it runs afoul of the .for/.if construction rules and this is needed to expand
>>> the for variable. It tripped me up when I looked at it, until I recalled a comment
>>> from similar code in NetBSD.
>>> 
>>> Warner
>>> 
>>> On Apr 21, 2014, at 7:15 AM, Ian Lepore <ian at FreeBSD.org> wrote:
>>> 
>>>> On Thu, 2014-03-27 at 20:44 +0100, Dimitry Andric wrote:
>>>>> On 27 Mar 2014, at 19:12, Jilles Tjoelker <jilles at stack.nl> wrote:
>>>>>> On Thu, Mar 27, 2014 at 11:05:00AM -0600, Warner Losh wrote:
>>>>>>> On Mar 26, 2014, at 4:30 PM, Dimitry Andric <dim at freebsd.org> wrote:
>>>>>>>> Author: dim
>>>>>>>> Date: Wed Mar 26 22:30:38 2014
>>>>>>>> New Revision: 263778
>>>>>>>> URL: http://svnweb.freebsd.org/changeset/base/263778
>>>>>> 
>>>>>>>> Log:
>>>>>>>> Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process
>>>>>>>> all the SUBDIR entries in parallel, instead of serially.  Apply this
>>>>>>>> option to a selected number of Makefiles, which can greatly speed up the
>>>>>>>> build on multi-core machines, when using make -j.
>>>>>> 
>>>>>>>> This can be extended to more Makefiles later on, whenever they are
>>>>>>>> verified to work correctly with parallel building.
>>>>>> 
>>>>>>> Why not have this ‘opt out’ rather than ‘opt in’ like it is now? Are
>>>>>>> there any known bad dependencies this introduces?
>>>>>> 
>>>>>> I'm paranoid about build systems ;) It is easy to add dependencies
>>>>>> across directories and as long as directories are built in sequence,
>>>>>> nothing goes wrong.
>>>>>> 
>>>>>> In fact, I had enabled SUBDIR_PARALLEL in sys/modules/Makefile as well,
>>>>>> but this caused mysterious failures with some kernels such as mips
>>>>>> ADM5120.
>>>>> 
>>>>> There are a bunch of other parts that don't really like parallel builds
>>>>> at the moment.  For example, gnu/usr.bin/binutils needs its libraries
>>>>> (libbfd.a, etc) built first, before it can link the programs.  Similar
>>>>> for gnu/usr.bin/cc, which needs libiberty, libcpp, etc before being able
>>>>> to build the rest of gcc.
>>>>> 
>>>>> Most of these cases can hopefully be solved by adding .WAIT targets at
>>>>> strategic points in the SUBDIR lists, but this also needs a bit of extra
>>>>> logic in bsd.subdir.mk.
>>>>> 
>>>>> -Dimitry
>>>>> 
>>>> 
>>>> It turns out I needed the .WAIT functionality to use SUBDIR_PARALLEL for
>>>> $work, so I came up with the attached, does this look okay to commit?
>>>> 
>>>> -- Ian
>>>> 
>>>> diff -r 67802e319fc6 share/mk/bsd.subdir.mk
>>>> --- a/share/mk/bsd.subdir.mk	Sun Apr 20 21:01:07 2014 -0600
>>>> +++ b/share/mk/bsd.subdir.mk	Mon Apr 21 06:59:37 2014 -0600
>>>> @@ -4,10 +4,10 @@
>>>> # The include file <bsd.subdir.mk> contains the default targets
>>>> # for building subdirectories.
>>>> #
>>>> -# For all of the directories listed in the variable SUBDIRS, the
>>>> +# For all of the directories listed in the variable SUBDIR, the
>>>> # specified directory will be visited and the target made. There is
>>>> # also a default target which allows the command "make subdir" where
>>>> -# subdir is any directory listed in the variable SUBDIRS.
>>>> +# subdir is any directory listed in the variable SUBDIR.
>>>> #
>>>> #
>>>> # +++ variables +++
>>>> @@ -42,7 +42,7 @@ distribute:
>>>> 
>>>> _SUBDIR: .USE
>>>> .if defined(SUBDIR) && !empty(SUBDIR) && !defined(NO_SUBDIR)
>>>> -	@${_+_}for entry in ${SUBDIR}; do \
>>>> +	@${_+_}for entry in ${SUBDIR:N.WAIT}; do \
>>>> 		if test -d ${.CURDIR}/$${entry}.${MACHINE_ARCH}; then \
>>>> 			${ECHODIR} "===> ${DIRPRFX}$${entry}.${MACHINE_ARCH} (${.TARGET:realinstall=install})"; \
>>>> 			edir=$${entry}.${MACHINE_ARCH}; \
>>>> @@ -57,7 +57,7 @@ distribute:
>>>> 	done
>>>> .endif
>>>> 
>>>> -${SUBDIR}: .PHONY
>>>> +${SUBDIR:N.WAIT}: .PHONY
>>>> 	${_+_}@if test -d ${.TARGET}.${MACHINE_ARCH}; then \
>>>> 		cd ${.CURDIR}/${.TARGET}.${MACHINE_ARCH}; \
>>>> 	else \
>>>> @@ -65,13 +65,18 @@ distribute:
>>>> 	fi; \
>>>> 	${MAKE} all
>>>> 
>>>> +__wait=.WAIT
>>>> .for __target in all all-man checkdpadd clean cleandepend cleandir \
>>>>   depend distribute lint maninstall manlint \
>>>>   obj objlink realinstall regress tags \
>>>>   ${SUBDIR_TARGETS}
>>>> .ifdef SUBDIR_PARALLEL
>>>> +__subdir_targets=
>>>> .for __dir in ${SUBDIR}
>>>> -${__target}: ${__target}_subdir_${__dir}
>>>> +.if ${__wait} == ${__dir}
>>>> +__subdir_targets+= .WAIT
>>>> +.else
>>>> +__subdir_targets+= ${__target}_subdir_${__dir}
>>>> ${__target}_subdir_${__dir}: .MAKE
>>>> 	@${_+_}set -e; \
>>>> 		if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \
>>>> @@ -85,7 +90,9 @@ distribute:
>>>> 		fi; \
>>>> 		${MAKE} ${__target:realinstall=install} \
>>>> 		    DIRPRFX=${DIRPRFX}$$edir/
>>>> +.endif
>>>> .endfor
>>>> +${__target}: ${__subdir_targets}
>>>> .else
>>>> ${__target}: _SUBDIR
>>>> .endif
>>> 
>>> 
>>> 
>> 
>> 
> 
>