64-bit atomic ops on 32-bit CPU -- was: ZFS .. on PowerPC ?

[Rafal Jaworowski]

Marcel Moolenaar wrote:
>> a.) make sure the whole granule is dedicated for a single object we want
>> atomic access to, i.e. in case of a 64-bit object we would use only 2
>> words
>> out of 8 in the given granule and the remaining 6 would be wasted
> 
> Is this required for correctness? I can see that it is
> desirable for performance, but I don't think it's needed
> for correctness. Can you elaborate?

Having a dedicated granule for the object is not requirement. If this is not
met the reservation may be lost gratuitously due to other stores which might
happen within this granule (issued by other CPUs, and/or other threads on
local CPU doing stwcx in this granule). So this is would hit performance, as
you note, but the scheme would work still.

The strict requirement is having all words of the compound atomic object
within the same granule.

>> Atomic store skeleton (pseudo asm, but you'll get the idea):
>>
>> 1. lwarx W1
>>
>> 2. stw W2
>> This regular (non-stwcx) store issued from local CPU will not clear our
>> reservation on this granule (only external CPUs or other entities' stores
>> within this granule can clear it)
>>
>> 3. stwcx W1, goto p.1 if not succeeded
> 
> I don't see a read of W2. In particular, we clobber W2
> unconditionally, so we must guarantee that we always
> read the correct value of W2. Can you elaborate on how
> an increment would be made atomic?

This was an example of an atomic write when we don't care about the previous
state (there, the initial lwarx is only needed for obtaining the reservation).

Atomic increment would be like this:
1. lwarx W1

2. lwz W2
3. In-register increment/other modification
4. stw W2 (modified)
Neither 2. nor 3 issued from local CPU will not clear our reservation on this
granule.

(optionally increment/modify in-register value of W1)
5. stwcx W1, goto p.1 if not succeeded

If non-local CPU (or any other bus master) modifies anything within the
considered granule after we obtained the reservation in 1., the last stwcx
will fail and we'll loop again.

Rafal