Scaling and performance issues with FreeBSD 9 (& 10) on 4 socket systems

[Remy Nonnenmacher]

On 06/13/13 13:32, Mark Felder wrote:
> On Wed, 12 Jun 2013 17:58:49 -0500, David O'Brien <obrien at freebsd.org>
> wrote:
>
>> We found FreeBSD 8.4 to perform better than FreeBSD 9.1, and Linux
>> considerably better than both on the same machine.
>
> http://svnweb.freebsd.org/base?view=revision&revision=241246
>
> The above link is likely why 8.4 is better than 9.1 on the same machine.
>
>> We've tried various things and haven't been able to explain why FreeBSD
>> isn't scaling on the new hardware.  Nor why it performs so much worse
>> than FreeBSD on the older "M2" machines.
>
> The CPUs between those machines are quite different. I'm sure we're
> looking at different cache sizes, different behavior for the
> hyperthreading, etc. I'm sure others would be greatly interested in you
> providing the same benchmark results for a recent snapshot of HEAD as well.
> _______________________________________________
> freebsd-performance at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to
> "freebsd-performance-unsubscribe at freebsd.org"

We had same problem on 4x12 cores (AMD) machines. After investigating 
using hwpmc, it appears that performance was killed by a scheduler 
function trying to find "least used cpu" that unfortunately works on 
contended structures (ie: lots a cores are fighting to get works). A 
solution was found by using artificially long queue of stuck process 
(steal_thresh bumped to over 8) and by cpu affinity crafting.

Was a year ago and from my memory. I guess you may give a try to see if 
it helps.

Disregard is a scheduler specialist contradicts.

Thanks.