And Here I Thought buildworld/makeworld Was IO Bound

[Glen Barber]

On Mon, Feb 10, 2014 at 09:45:42AM -0600, Tim Daneliuk wrote:
> >>Well ... it has to hit the disk sooner or later.  But, if the
> >>frequency of physical writes is low because of aggregated
> >>IO from the cache, I guess that would tend to make the whole
> >>business more CPU bound than IO bound.  I just found this surprising.
> >>
> >
> >What is the underlying filesystem?
> >
> 
> UFS w/Softupdates, no journaling.
> 
> 
> [...]

> >>Yeah, I wonder what other people are seeing for a full buildworld/kernel and/or what
> >>the master machines at FreeBSD.org do in this regards.
> >>
> >>Would anyone else care to share with the class?
> >>
> >
> >If you mean "what do the machines do with regard to tuning", the only
> >specific tuning is turning off atime.
> >
> 
> 
> Could you comment a bit more about this please?  How you do it,
> rationale', etc.
> 

Each build happens in its own ZFS dataset, so atime is disabled during
dataset creation (zfs create -o atime=off zroot/11-amd64-GENERIC-snap).

The tuning(7) manual has more details, but basically each time a file is
read, a separate write happens to update the file access time.  Turning
off atime eliminates these excessive writes.

> >If you mean "what do the machines do as far as overall build time", with
> >a clean obj/ directory, 35 minutes sounds about right.  I do not have
> >exact numbers.
> >
> >The machine that currently produces weekly snapshot images uses '-j10'
> >for buildworld and '-j6' for buildkernel for each build.  It also runs
> >the builds in parallel.  Even with three parallel 'make -j10 buildworld'
> >(one for head/, stable/10/, and stable/9/), disk IO is minimal.  The
> >only time the disk IO becomes a bottleneck is when creating the
> >distribution files (base.txz, kernel.txz, etc.).
> 
> 
> What sort of CPU/Mem/Disk is that machine?
> 

This machine is 48-core (4 x 12 cores) Opteron 6174 (2.2GHz), 128GB RAM,
with 5 drives in a raidz1 on a PERC H700 controller.

Glen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20140210/db757540/attachment.sig>