svn commit: r253002 - head

[Garrett Cooper]

On Mon, Jul 8, 2013 at 2:13 PM, John Baldwin <jhb at freebsd.org> wrote:
> On Monday, July 08, 2013 2:23:31 am Garrett Cooper wrote:
>> On Sun, Jul 7, 2013 at 7:25 PM, Garrett Cooper <yaneurabeya at gmail.com>
> wrote:
>> > On Jul 7, 2013, at 2:15 PM, Alfred Perlstein <alfred at freebsd.org> wrote:
>> >
>> >> On 7/7/13 2:01 PM, Garrett Cooper wrote:
>> >>> Why the magic number 12?
>> >>
>> >> Numbers higher seem to result in worse performance as reported by some
> members of my team.
>> >
>> > The suggestion is good in spirit, but this doesn't justify the reasoning
> for this recommendation for all cases.
>> >
>> > Please revert this change and add a doc page or notes to the dev handbook
> discussing what the empirical process and results were for determining this
> value so people can come up with their own values that work best with their
> hardware and software config. This recommendation is prone to bitrot like some
> of the recommendations in tuning(7).
>> >
>> > Misinformation is sometimes more harmful than no information.
>>
>> I spoke with Alfred over the phone and did some more careful thought
>> about this and I'm rescinding this request.
>>
>> Alfred did a good job at documenting how JFLAG works (it was
>> previously undocumented). My concern over -j12 was performance
>> related, and after giving things more careful thought it actually
>> makes sense why -j12 was chosen because Westmere and newer processors
>> have issues with NUMA and cache locality between multiple processor
>> packages as we've seen non-empirically and empirically at Isilon with
>> FreeBSD 7 and 10 (it's a known issue that jeffr@ and jhb@ are aware
>> of).
>>
>> I'll come up with a concise patch that does what Alfred was trying to
>> achieve and have Alfred review it.
>>
>> Thanks (and thank you Alfred for the contribution!!!)!
>
> Westmere is fine, it's post-Westmere that is more troublesome.

Even the 6-core Westmeres (I'm being completely dumb here as you and
Jeff know a lot more about the NUMA issue than I do as I just caught
the tail end of the conversation at BSDCan)? I'm asking because they
(iX) are using build.ix as the primary build machine and it has 2
Westmere dies with (IIRC -- please correct me if I'm wrong
Alfred/Xin/etc) 6 cores each and are SMT enabled. It also has a
boatload of RAM and disks hooked up to an mfi(4) controller (which
could be a contributing factor in the performance degradation issue).

> I think the comment is not super useful, but don't object enough to want
> it to be removed.  I always use 'make tinderbox' instead of
> 'make universe' though as I want build failures to be obvious.  For the
> described use case of "checking if kernels build", 'tinderbox' certainly
> seems to be the more appropriate target.

Changing it from universe to tinderbox seems like a better idea --
I'll put a short note in my proposed patch for that.

Thanks!
-Garrett