And Here I Thought buildworld/makeworld Was IO Bound

[Tim Daneliuk]

On 02/10/2014 09:24 AM, Glen Barber wrote:
> On Mon, Feb 10, 2014 at 08:57:10AM -0600, Tim Daneliuk wrote:
>> On 02/09/2014 11:45 PM, army.of.root wrote:
>>> Am 10/02/14 04:43, schrieb Tim Daneliuk:
>>>> For some years now, I have been doing nightly builds of -STABLE
>>>> on an old Pentium D machine with 2G of memory.  Buildworld + 2
>>>> different kernels was taking in the neighborhood of 3 1/2 hours or so
>>>> to run.
>>>>
>>>> I then upgraded the Mobo/CPU to a Haswell Quadcore I5-4570 and, sure
>>>> enough, the build time for all the above came down to 30-35 mins or so.
>>>>
>>>> "So", says I, "I'll bet a faster drive would help considering all the
>>>> scribbling to the disk the compilers and makes do".  So, I upgdared to
>>>> a Kingston SSD Now 300, 120G hard drive and he time to do the above
>>>> went down to .... wait, it's still about 30-35 mins ????
>>>>
>>>> So, I've tried fiddling with different values for -j on the make
>>>> command line to little avail.  Well, -j8 and -j16 show no real
>>>> difference here.
>>>>
>>>> So is the bounding function here actually CPU not IO?  Am I missing
>>>> something?
>>>>
>>>> Thanks,
>>>>
>>>> P.S.  Trying now with no -j arg on make invocation.
>>>
>>> Hi,
>>>
>>> the new machine has a lot more memory, I assume.
>>
>> Yes, 8G as opposed to 2G. I suppose I could bump it out
>> to 16G and see if this makes any material difference with
>> even more cache space.
>>
>>> The build probably will not even hit the disk due to caching.
>>
>>
>> Well ... it has to hit the disk sooner or later.  But, if the
>> frequency of physical writes is low because of aggregated
>> IO from the cache, I guess that would tend to make the whole
>> business more CPU bound than IO bound.  I just found this surprising.
>>
>
> What is the underlying filesystem?
>

UFS w/Softupdates, no journaling.


>>>
>>> Also remember, the build process spawns probably millions of processes and that alone takes some time.
>>>
>>> And 30min sounds pretty great to me :D
>>
>> Yeah, I wonder what other people are seeing for a full buildworld/kernel and/or what
>> the master machines at FreeBSD.org do in this regards.
>>
>> Would anyone else care to share with the class?
>>
>
> If you mean "what do the machines do with regard to tuning", the only
> specific tuning is turning off atime.
>


Could you comment a bit more about this please?  How you do it,
rationale', etc.

> If you mean "what do the machines do as far as overall build time", with
> a clean obj/ directory, 35 minutes sounds about right.  I do not have
> exact numbers.
>
> The machine that currently produces weekly snapshot images uses '-j10'
> for buildworld and '-j6' for buildkernel for each build.  It also runs
> the builds in parallel.  Even with three parallel 'make -j10 buildworld'
> (one for head/, stable/10/, and stable/9/), disk IO is minimal.  The
> only time the disk IO becomes a bottleneck is when creating the
> distribution files (base.txz, kernel.txz, etc.).


What sort of CPU/Mem/Disk is that machine?

>
> Glen
>


-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/