make -j 4 is really make -j 8 for buildworld?

Scott Long scottl at freebsd.org
Tue Jan 13 21:33:40 PST 2004 

Gordon and I sat down in September and tried to track it down since it
seemed to be related to building /rescue.  Unfortunately, we didn't seem
to get anywhere.  Try building without /rescue and see if it makes a
difference.

Scott


Robert Watson wrote:
> A few times in the past, I've noticed that when I run 'make -j4
> buildworld', I frequently run with load averages between 8 and 10 on one
> of my boxes at work.  Recently, I actually sat around with ps -ax for a
> bit, and found that frequently, 8 different compiler/assembler instances
> are running at once, rather than the four I expect:
> 
> cboss:/data/p4/rwatson/trustedbsd/audit2/sys/i386/compile/AUDIT> ps ax | grep make ; ps ax | grep cc ; ps ax | grep as
> 22892  p2  S      0:01.92 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make -f Makefile.inc1 par-all
> 23772  p2  S      0:01.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/
> 23992  p2  S      0:01.04 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/
> 39254  p2  S      0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/
> 39265  p2  S      0:01.41 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/cc_int/
> 40689  p2  S      0:10.78 make -j 4 buildworld
> 40717  p2  S      0:06.36 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make -m /data/p4/rwatson/trustedbsd/mac/share/mk -f Makefile.inc1 buildworld
> 42307  p2  S      0:00.30 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.bin/
> 43220  p2  S      0:00.21 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.sbin/
> 45194  p2  R      0:00.06 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.bin/indent/
> 45217  p2  R      0:00.07 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.sbin/config/
> ...
> 39254  p2  S      0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/
> 39265  p2  S      0:01.41 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/cc_int/
> 45169  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
> 45170  p2  R      0:01.51 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
> 45198  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -c /data/p4/rwatson/trustedbsd/mac/usr.bin/indent/indent.c
> 45199  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -c /data/p4/rwatson/trustedbsd/mac/usr.bin/indent/io.c
> 45201  p2  R      0:00.77 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEVEL__=3 -D_LONGLONG /data/p4/rwatson/trustedbsd/mac/usr
> 45204  p2  R      0:00.79 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEVEL__=3 -D_LONGLONG /data/p4/rwatson/trustedbsd/mac/usr
> 45212  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
> 45213  p2  R      0:01.12 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
> 45231  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/main.c
> 45232  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c config.c
> 45233  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/mkmakefile.c
> 45234  p2  R      0:00.42 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
> 45236  p2  R      0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
> 45239  p2  R      0:00.42 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
> 45241  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/mkheaders.c
> 45242  p2  R      0:00.23 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
> 45245  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
> 45246  p2  R      0:00.50 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
> 45249  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
> 45250  p2  R      0:00.04 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
> ...
> 45171  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgcleanup.o -
> 45202  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o io.o -
> 45205  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o indent.o -
> 45214  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfglayout.o -
> 45235  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o config.o -
> 45237  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o main.o -
> 45240  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o mkmakefile.o -
> 45243  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o mkheaders.o -
> 45247  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgloop.o -
> 45251  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgrtl.o -
> 
> I'm all for parallelism, but there's "useful parallelism" and "excessive
> parallelism".  Is this the intended behavior of make -jX, or am I running
> into a bit shifting bug?  Is it just our buildworld bits losing some state
> on the way through and trying a bit too hard to passing on the make
> arguments? 
> 
> Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
> robert at fledge.watson.org      Senior Research Scientist, McAfee Research
> 
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
>

More information about the freebsd-current mailing list