make -j 4 is really make -j 8 for buildworld?
Robert Watson
rwatson at FreeBSD.org
Tue Jan 13 21:23:56 PST 2004
A few times in the past, I've noticed that when I run 'make -j4
buildworld', I frequently run with load averages between 8 and 10 on one
of my boxes at work. Recently, I actually sat around with ps -ax for a
bit, and found that frequently, 8 different compiler/assembler instances
are running at once, rather than the four I expect:
cboss:/data/p4/rwatson/trustedbsd/audit2/sys/i386/compile/AUDIT> ps ax | grep make ; ps ax | grep cc ; ps ax | grep as
22892 p2 S 0:01.92 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make -f Makefile.inc1 par-all
23772 p2 S 0:01.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/
23992 p2 S 0:01.04 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/
39254 p2 S 0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/
39265 p2 S 0:01.41 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/cc_int/
40689 p2 S 0:10.78 make -j 4 buildworld
40717 p2 S 0:06.36 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make -m /data/p4/rwatson/trustedbsd/mac/share/mk -f Makefile.inc1 buildworld
42307 p2 S 0:00.30 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.bin/
43220 p2 S 0:00.21 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.sbin/
45194 p2 R 0:00.06 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.bin/indent/
45217 p2 R 0:00.07 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.sbin/config/
...
39254 p2 S 0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/
39265 p2 S 0:01.41 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/cc_int/
45169 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
45170 p2 R 0:01.51 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
45198 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -c /data/p4/rwatson/trustedbsd/mac/usr.bin/indent/indent.c
45199 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -c /data/p4/rwatson/trustedbsd/mac/usr.bin/indent/io.c
45201 p2 R 0:00.77 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEVEL__=3 -D_LONGLONG /data/p4/rwatson/trustedbsd/mac/usr
45204 p2 R 0:00.79 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEVEL__=3 -D_LONGLONG /data/p4/rwatson/trustedbsd/mac/usr
45212 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
45213 p2 R 0:01.12 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
45231 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/main.c
45232 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c config.c
45233 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/mkmakefile.c
45234 p2 R 0:00.42 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
45236 p2 R 0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
45239 p2 R 0:00.42 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
45241 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/mkheaders.c
45242 p2 R 0:00.23 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
45245 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
45246 p2 R 0:00.50 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
45249 p2 S 0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
45250 p2 R 0:00.04 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
...
45171 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgcleanup.o -
45202 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o io.o -
45205 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o indent.o -
45214 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfglayout.o -
45235 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o config.o -
45237 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o main.o -
45240 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o mkmakefile.o -
45243 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o mkheaders.o -
45247 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgloop.o -
45251 p2 S 0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgrtl.o -
I'm all for parallelism, but there's "useful parallelism" and "excessive
parallelism". Is this the intended behavior of make -jX, or am I running
into a bit shifting bug? Is it just our buildworld bits losing some state
on the way through and trying a bit too hard to passing on the make
arguments?
Robert N M Watson FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org Senior Research Scientist, McAfee Research
More information about the freebsd-current
mailing list