32-bit powerpc: tlbie context-synchronizing needs (according to MPCFPE32B.pdf) vs. FreeBSD's 32-bit powerpc kernel code

[Mark Millard]

On 2019-Apr-23, at 20:46, Mark Millard <marklmi at yahoo.com> wrote:

> In looking into the iMac G3 (MPC750) failures I ran into
> the following in the 32-bit powerpc FreeBSD code. It
> may be that MPC750's are just not intended to be
> supported any more. Later in this note I quote MPCFPE32B.pdf
> about the criteria for tlbie use in the 32-bit context.
> 
> Looking at the tlbie() code I saw (for a non-debug
> 32-bit-powerpc kernel build):
> 
> <tlbie+0xa4> bl      0054d4e4 <_mtx_lock_spin_cookie>
> <tlbie+0xa8> ptesync
> <tlbie+0xac> tlbie   r28
> <tlbie+0xb0> eieio
> <tlbie+0xb4> tlbsync
> <tlbie+0xb8> ptesync
> 
> ptesync is not even defined or mentioned in MPCFPE32B.pdf 
> or MPC750UM.pdf . (I did not find any examples of lwsync
> in the 32-bit kernel. It would also have an undefined
> status on an MPC750.)
> 
> All other ptesync usage was in routines with names
> that start with moea64_ . But tlbie(. . .) is used from
> the likes of moea_clear_bit, moea_cpu_bootstrap,
> moea_page_set_memattr, moea_pvo_remove, moea_remove_write,
> moea_protect, moea_pte_insert, moea_pte_spill, and
> moea_mapdev_attr. SO the ptesync usage in 32-bit code
> is fairly extensive.
> 
> ptesync seems to have an instruction encoding that is
> reserved according to MPCFPE32B.pdf and MPC750UM.pdf .
> eieio documented but not listed as context-synchronizing.
> But it orders the tlbie vs. tlbsync: that is documented.
> sync is not listed as context-synchronizing either. (This
> matters later.)
> 
> _mtx_lock_spin_cookie returning does a bunch of memory
> reads (at least from the stack) after any isync it might
> execute. When prior code skips _mtx_lock_spin_cookie,
> there is an isync involved.
> 
> The MPCFPE32B.pdf Data Access Synchronization table
> indicates that for tlbie:
> 
> Required Prior: Context-synchronizing
> Required After: Context-synchronizing or sync
> 
> It also has a note (not covering all the multiprocessor
> system issues, as noted in the note after this one):
> 
> QUOTE
> For data accesses, the context synchronizing instruction before the tlbie, or tlbia instruction ensures that all
> memory accesses, due to preceding instructions, have completed to a point at which they have reported all interrupts
> that may be caused. The context synchronizing instruction after the tlbie, or tlbia ensures that subsequent memory
> accesses will not use the TLB entries being invalidated. It does not ensure that all memory accesses previously
> translated by the TLB entries being invalidated have completed with respect to memory or, for tlbie or tlbia, that R
> and C bit updates associated with those memory accesses have completed; if these completions must be ensured, the
> tlbie, or tlbia must be followed by a sync instruction rather than by a context synchronizing instruction.
> END QUOTE
> 
> 
> So it looks to me like if the MPC750 (and, so, iMac G3)
> is to be supported, there should (also?) be an isync
> before the tlbie.
> 
> As for after the tlbie, the code is supposed to work
> for multiprocessors so sync would appear to be required
> by the note (to be sure tlbsync completed). But,
> for the self-same (or only) processor, it would appear
> that the "does not ensure that all memory accesses . . ."
> material means that an isync is possibly also necessary.

I found material indicating that even the uniprocessor
system designs need the sync instructions:

QUOTE
The sync instructions shown are required even for single processor systems (to ensure that all previous
changes to the page tables and all preceding tlbie instructions have completed).
END QUOTE

This was for examples of  how to add a Page Table Entry
and how to delete one. I assume that the above is because
of the "if these completions must be ensured" issue.

This makes it seem that the "rather than by a context
synchronizing instruction" in the bigger quote overrides
the earlier part about "context synchronizing instruction
after the tlbie".

That still leaves having a isync before the tlbie,
as well as the sync vs ptesync issue.


> It is not clear what the ptesync instruction encoding
> leads to on the MPC750. Is it treated as MPC750 sync?
> As a no-op? As something else? The MPCFPE32B.pdf and
> MPC750UM.pdf indicate that sync is appropriate (with
> the L=0 encoding specifically, in more modern terms).
> 
> 
> Of course, allowing 32-bit FreeBSD to run on powerpc64
> hardware may well not be able to use just the 32-bit's
> MPCFPE32B.pdf and MPC750UM.pdf criteria related to tlbie.
> This may well be more important than iMac G3's or other
> MPC750 like contexts.
> 




===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)