make -jN test

[pete wright]

(warning long post)
Hi all,
     A recent thread on a I subscribe to in regard to the usefullness
of using the -j flag on a make world led me to run a test over the
weekend just to see it's effectiveness.  This is the section of the
handbook that started it all for me:
(from: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html)

"It is now possible to specify a -j option to make which will cause it
to spawn several simultaneous processes. This is most useful on
multi-CPU machines. However, since much of the compiling process is IO
bound rather than CPU bound it is also useful on single CPU machines.

On a typical single-CPU machine you would run:

# make -j4 buildworld

make(1) will then have up to 4 processes running at any one time.
Empirical evidence posted to the mailing lists shows this generally
gives the best performance benefit.

If you have a multi-CPU machine and you are using an SMP configured
kernel try values between 6 and 10 and see how they speed things up."

So to test this out I wrote a shell script to time a make -jN
buildworld where N = a number from 1-8.  The idea is to see if there
is any speedup between running make -j1 and make -j8 on a dual proc
box.  Now this is not scientific, altho maybe the findings are typical
for a low system load box.  I also had the box sleep for 60 secs
between each build. Here are my timings:

building world with 1 jobs...

real    95m20.543s
user    83m18.550s
sys     13m20.120s

cleaning /usr/src....
building world with 2 jobs...

real    54m15.856s
user    84m42.337s
sys     16m39.216s

cleaning /usr/src....
building world with 3 jobs...

real    53m53.239s
user    85m12.189s
sys     17m13.039s

cleaning /usr/src....
building world with 4 jobs...

real    53m56.539s
user    85m22.767s
sys     17m22.433s

cleaning /usr/src....
building world with 5 jobs...

real    54m10.993s
user    85m30.995s
sys     17m14.160s

cleaning /usr/src....
building world with 6 jobs...

real    54m10.814s
user    85m24.063s
sys     17m27.437s

cleaning /usr/src....
building world with 7 jobs...

real    54m15.289s
user    85m25.479s
sys     17m27.751s

cleaning /usr/src....
building world with 8 jobs...

real    54m23.526s
user    85m27.173s
sys     17m28.891s

As you can tell there does not seem to be a rampup after a make -j2
buildworld.  This is a SMP system, with stripped scisi disks (altho
the controller is GIANT locked....maybe this is preventing additional
gains?).  For completness here is my dmesg:

Copyright (c) 1992-2005 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD 5.3-STABLE #1: Tue Jan  4 19:06:52 EST 2005
    root at host.mydomain.org:/usr/obj/usr/src/sys/SMP
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Pentium(R) III CPU - S         1266MHz (1263.08-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0x6b4  Stepping = 4
  Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE>
real memory  = 1610530816 (1535 MB)
avail memory = 1568518144 (1495 MB)
MPTable: <IBM ENSW NF 4100R SMP>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 cpu0 (BSP): APIC ID:  3
 cpu1 (AP): APIC ID:  0
ioapic0: Assuming intbase of 0
ioapic1: Assuming intbase of 16
ioapic1 <Version 1.1> irqs 16-31 on motherboard
ioapic0 <Version 1.1> irqs 0-15 on motherboard
npx0: [FAST]
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <MPTable Host-PCI bridge> pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
pci0: <display, VGA> at device 1.0 (no driver attached)
fxp0: <Intel 82559 Pro/100 Ethernet> port 0x2200-0x223f mem
0xfea00000-0xfeafffff,0xfeb7f000-0xfeb7ffff irq 27 at device 2.0 on
pci0
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> on miibus0
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:09:6b:9a:12:13
fxp1: <Intel 82559 Pro/100 Ethernet> port 0x2240-0x227f mem
0xfe900000-0xfe9fffff,0xfeb7e000-0xfeb7efff irq 25 at device 10.0 on
pci0
miibus1: <MII bus> on fxp1
inphy1: <i82555 10/100 media interface> on miibus1
inphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: Ethernet address: 00:09:6b:9a:12:14
isab0: <PCI-ISA bridge> port 0x440-0x44f at device 15.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <ServerWorks ROSB4 UDMA33 controller> port
0x700-0x70f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 15.1 on pci0
ata0: channel #0 on atapci0
ata1: channel #1 on atapci0
ohci0: <OHCI (generic) USB controller> mem 0xfeb7d000-0xfeb7dfff irq 7
at device 15.2 on pci0
ohci0: [GIANT-LOCKED]
usb0: OHCI version 1.0, legacy support
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: (0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pcib1: <MPTable Host-PCI bridge> pcibus 1 on motherboard
pci1: <PCI bus> on pcib1
pci1: <mass storage, SCSI> at device 3.0 (no driver attached)
ips0: <IBM ServeRAID Adapter> mem 0xedffe000-0xedffffff irq 20 at
device 5.0 on pci1
ips0: [GIANT-LOCKED]
cpu0 on motherboard
cpu1 on motherboard
orm0: <ISA Option ROMs> at iomem
0xcc800-0xd47ff,0xcb000-0xcc7ff,0xc0000-0xcafff on isa0
pmtimer0 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x64,0x60 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
fdc0: <Enhanced floppy controller> at port 0x3f0-0x3f5 irq 6 drq 2 on isa0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
unknown: <PNP0501> can't assign resources (port)
ppc1: parallel port not found.
unknown: <PNP0700> can't assign resources (port)
unknown: <PNP0303> can't assign resources (port)
Timecounters tick every 10.000 msec
acd0: CDROM <LG CD-ROM CRN-8245B/1.16> at ata0-master UDMA33
ips0: adapter type: ServeRAID 4Lx (neo lite)
ips0: logical drives: 1
ips0: Logical Drive 0: RAID1 sectors: 71096320, state OK
ipsd0: <Logical Drive> on ips0
ipsd0: Logical Drive  (34715MB)
SMP: AP CPU #1 Launched!
Mounting root from ufs:/dev/ipsd0s1a

So, unless I misread the section of the handbook, I should be seeing a
decrease in time for these buldworlds but am not.  Any idea why this
is not happening.

Cheers,
     Pete Wright
-- 
~~o0OO0o~~
Pete Wright
www.nycbug.org
NYC's *BSD User Group