New SCHED_SMP diff.

[Robert Watson]

On Mon, 2 Jul 2007, Suleiman Souhlal wrote:

>> I don't think here you need an atomic instruction, a memory barrier throug 
>> sfence is good enough in order to make thread migration consistent.
>
> SFENCE is not needed. Stores are already strongly ordered wrt other stores 
> on x86 (unless you use write-combining memory or non-temporal stores). The 
> main advantage of using an atomic operation when unlocking is that it should 
> make the store visible to other CPUs faster (so they don't spin as long), 
> although I think you'll have a hard time noticing a difference in a 
> macrobenchmark.

FYI, in my experience the difference between using an atomic operation and a 
non-atomic operation to force out the release of a lock *is* measurable in a 
macrobenchmark.  I measured a several percentage slowdown in buildworld time 
when I made the mutex release use a non-atomic operation, even though the 
cycle count for the release operation went up, which I put down to additional 
waiting time across several CPUs.  Our kernel was quite different then -- much 
less fine-grained locking, such as Giant over VFS still -- and what is now 
several-year old SMP hardware.

It would be very interesting to re-run micro- and macrobenchmarks to continue 
to reevaluate some of the decisions we made a few years ago on that generation 
of MP hardware and with the incomplete locking work now that MP hardware has 
changed a lot and our locking is much more mature.

Robert N M Watson
Computer Laboratory
University of Cambridge