svn commit: r263778 - in head: bin lib lib/clang sbin share/mk usr.bin usr.sbin

[Ian Lepore]

It's also a bit odd that the way of opting in makes it appear as if
you're opting out.

-- Ian

On Thu, 2014-03-27 at 11:05 -0600, Warner Losh wrote:
> Why not have this ‘opt out’ rather than ‘opt in’ like it is now? Are there any known bad dependencies this introduces?
> 
> Warner
> 
> On Mar 26, 2014, at 4:30 PM, Dimitry Andric <dim at freebsd.org> wrote:
> 
> > Author: dim
> > Date: Wed Mar 26 22:30:38 2014
> > New Revision: 263778
> > URL: http://svnweb.freebsd.org/changeset/base/263778
> > 
> > Log:
> >  Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process
> >  all the SUBDIR entries in parallel, instead of serially.  Apply this
> >  option to a selected number of Makefiles, which can greatly speed up the
> >  build on multi-core machines, when using make -j.
> > 
> >  This can be extended to more Makefiles later on, whenever they are
> >  verified to work correctly with parallel building.
> > 
> >  I tested this on a 24-core machine, with make -j48 buildworld (N = 6):
> > 
> >                  before    stddev       after    stddev
> >                  =======   ======       =======  ======
> >  real time        1741.1     16.5         959.8     2.7
> >  user time       12468.7     16.4       14393.0    16.8
> >  sys  time        1825.0     54.8        2110.6    22.8
> > 
> >  (user+sys)/real     8.2                   17.1
> > 
> >  E.g. the build was approximately 45% faster in real time.  On machines
> >  with less cores, or with lower -j settings, the speedup will not be as
> >  impressive.  But at least you can now almost max out a machine with
> >  buildworld!
> > 
> >  Submitted by:	jilles
> >  MFC after:	2 weeks
> > 
> > Modified:
> >  head/bin/Makefile
> >  head/lib/Makefile
> >  head/lib/clang/Makefile
> >  head/sbin/Makefile
> >  head/share/mk/bsd.subdir.mk
> >  head/usr.bin/Makefile
> >  head/usr.sbin/Makefile
> > 
> > Modified: head/bin/Makefile
> > ==============================================================================
> > --- head/bin/Makefile	Wed Mar 26 20:43:40 2014	(r263777)
> > +++ head/bin/Makefile	Wed Mar 26 22:30:38 2014	(r263778)
> > @@ -60,4 +60,6 @@ SUBDIR+=	tests
> > 
> > SUBDIR:=	${SUBDIR:O}
> > 
> > +SUBDIR_PARALLEL=
> > +
> > .include <bsd.subdir.mk>
> > 
> > Modified: head/lib/Makefile
> > ==============================================================================
> > --- head/lib/Makefile	Wed Mar 26 20:43:40 2014	(r263777)
> > +++ head/lib/Makefile	Wed Mar 26 22:30:38 2014	(r263778)
> > @@ -276,4 +276,8 @@ afterinstall:
> > 	${INSTALL_SYMLINK} ../include ${DESTDIR}/usr/lib/include
> > .endif
> > 
> > +.if !make(install)
> > +SUBDIR_PARALLEL=
> > +.endif
> > +
> > .include <bsd.subdir.mk>
> > 
> > Modified: head/lib/clang/Makefile
> > ==============================================================================
> > --- head/lib/clang/Makefile	Wed Mar 26 20:43:40 2014	(r263777)
> > +++ head/lib/clang/Makefile	Wed Mar 26 22:30:38 2014	(r263778)
> > @@ -147,4 +147,6 @@ SUBDIR+=liblldb \
> > 
> > SUBDIR+= include
> > 
> > +SUBDIR_PARALLEL=
> > +
> > .include <bsd.subdir.mk>
> > 
> > Modified: head/sbin/Makefile
> > ==============================================================================
> > --- head/sbin/Makefile	Wed Mar 26 20:43:40 2014	(r263777)
> > +++ head/sbin/Makefile	Wed Mar 26 22:30:38 2014	(r263778)
> > @@ -126,4 +126,6 @@ SUBDIR+=	tests
> > 
> > SUBDIR:=	${SUBDIR:O}
> > 
> > +SUBDIR_PARALLEL=
> > +
> > .include <bsd.subdir.mk>
> > 
> > Modified: head/share/mk/bsd.subdir.mk
> > ==============================================================================
> > --- head/share/mk/bsd.subdir.mk	Wed Mar 26 20:43:40 2014	(r263777)
> > +++ head/share/mk/bsd.subdir.mk	Wed Mar 26 22:30:38 2014	(r263778)
> > @@ -71,7 +71,26 @@ ${SUBDIR}: .PHONY .MAKE
> > .for __target in all all-man checkdpadd clean cleandepend cleandir \
> >     cleanilinks depend distribute lint maninstall manlint obj objlink \
> >     realinstall regress tags ${SUBDIR_TARGETS}
> > +.ifdef SUBDIR_PARALLEL
> > +.for __dir in ${SUBDIR}
> > +${__target}: ${__target}_subdir_${__dir}
> > +${__target}_subdir_${__dir}: .MAKE
> > +	@${_+_}set -e; \
> > +		if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \
> > +			${ECHODIR} "===> ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__target:realinstall=install})"; \
> > +			edir=${__dir}.${MACHINE_ARCH}; \
> > +			cd ${.CURDIR}/$${edir}; \
> > +		else \
> > +			${ECHODIR} "===> ${DIRPRFX}${__dir} (${__target:realinstall=install})"; \
> > +			edir=${__dir}; \
> > +			cd ${.CURDIR}/$${edir}; \
> > +		fi; \
> > +		${MAKE} ${__target:realinstall=install} \
> > +		    DIRPRFX=${DIRPRFX}$$edir/
> > +.endfor
> > +.else
> > ${__target}: _SUBDIR
> > +.endif
> > .endfor
> > 
> > .for __target in files includes
> > 
> > Modified: head/usr.bin/Makefile
> > ==============================================================================
> > --- head/usr.bin/Makefile	Wed Mar 26 20:43:40 2014	(r263777)
> > +++ head/usr.bin/Makefile	Wed Mar 26 22:30:38 2014	(r263778)
> > @@ -379,4 +379,6 @@ SUBDIR+=	svn
> > 
> > SUBDIR:=	${SUBDIR:O}
> > 
> > +SUBDIR_PARALLEL=
> > +
> > .include <bsd.subdir.mk>
> > 
> > Modified: head/usr.sbin/Makefile
> > ==============================================================================
> > --- head/usr.sbin/Makefile	Wed Mar 26 20:43:40 2014	(r263777)
> > +++ head/usr.sbin/Makefile	Wed Mar 26 22:30:38 2014	(r263778)
> > @@ -320,4 +320,6 @@ SUBDIR+=	wpa
> > 
> > SUBDIR:=	${SUBDIR:O}
> > 
> > +SUBDIR_PARALLEL=
> > +
> > .include <bsd.subdir.mk>
> > 
> 
>