And Here I Thought buildworld/makeworld Was IO Bound

[Glen Barber]

On Mon, Feb 10, 2014 at 08:57:10AM -0600, Tim Daneliuk wrote:
> On 02/09/2014 11:45 PM, army.of.root wrote:
> >Am 10/02/14 04:43, schrieb Tim Daneliuk:
> >>For some years now, I have been doing nightly builds of -STABLE
> >>on an old Pentium D machine with 2G of memory.  Buildworld + 2
> >>different kernels was taking in the neighborhood of 3 1/2 hours or so
> >>to run.
> >>
> >>I then upgraded the Mobo/CPU to a Haswell Quadcore I5-4570 and, sure
> >>enough, the build time for all the above came down to 30-35 mins or so.
> >>
> >>"So", says I, "I'll bet a faster drive would help considering all the
> >>scribbling to the disk the compilers and makes do".  So, I upgdared to
> >>a Kingston SSD Now 300, 120G hard drive and he time to do the above
> >>went down to .... wait, it's still about 30-35 mins ????
> >>
> >>So, I've tried fiddling with different values for -j on the make
> >>command line to little avail.  Well, -j8 and -j16 show no real
> >>difference here.
> >>
> >>So is the bounding function here actually CPU not IO?  Am I missing
> >>something?
> >>
> >>Thanks,
> >>
> >>P.S.  Trying now with no -j arg on make invocation.
> >
> >Hi,
> >
> >the new machine has a lot more memory, I assume.
> 
> Yes, 8G as opposed to 2G. I suppose I could bump it out
> to 16G and see if this makes any material difference with
> even more cache space.
> 
> >The build probably will not even hit the disk due to caching.
> 
> 
> Well ... it has to hit the disk sooner or later.  But, if the
> frequency of physical writes is low because of aggregated
> IO from the cache, I guess that would tend to make the whole
> business more CPU bound than IO bound.  I just found this surprising.
> 

What is the underlying filesystem?

> >
> >Also remember, the build process spawns probably millions of processes and that alone takes some time.
> >
> >And 30min sounds pretty great to me :D
> 
> Yeah, I wonder what other people are seeing for a full buildworld/kernel and/or what
> the master machines at FreeBSD.org do in this regards.
> 
> Would anyone else care to share with the class?
> 

If you mean "what do the machines do with regard to tuning", the only
specific tuning is turning off atime.

If you mean "what do the machines do as far as overall build time", with
a clean obj/ directory, 35 minutes sounds about right.  I do not have
exact numbers.

The machine that currently produces weekly snapshot images uses '-j10'
for buildworld and '-j6' for buildkernel for each build.  It also runs
the builds in parallel.  Even with three parallel 'make -j10 buildworld'
(one for head/, stable/10/, and stable/9/), disk IO is minimal.  The
only time the disk IO becomes a bottleneck is when creating the
distribution files (base.txz, kernel.txz, etc.).

Glen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20140210/a95fc4db/attachment.sig>