32-bit powerpc: tlbie context-synchronizing needs (according to MPCFPE32B.pdf) vs. FreeBSD's 32-bit powerpc kernel code

[Mark Millard]

In looking into the iMac G3 (MPC750) failures I ran into
the following in the 32-bit powerpc FreeBSD code. It
may be that MPC750's are just not intended to be
supported any more. Later in this note I quote MPCFPE32B.pdf
about the criteria for tlbie use in the 32-bit context.

Looking at the tlbie() code I saw (for a non-debug
32-bit-powerpc kernel build):

 <tlbie+0xa4> bl      0054d4e4 <_mtx_lock_spin_cookie>
 <tlbie+0xa8> ptesync
 <tlbie+0xac> tlbie   r28
 <tlbie+0xb0> eieio
 <tlbie+0xb4> tlbsync
 <tlbie+0xb8> ptesync

ptesync is not even defined or mentioned in MPCFPE32B.pdf 
or MPC750UM.pdf . (I did not find any examples of lwsync
in the 32-bit kernel. It would also have an undefined
status on an MPC750.)

All other ptesync usage was in routines with names
that start with moea64_ . But tlbie(. . .) is used from
the likes of moea_clear_bit, moea_cpu_bootstrap,
moea_page_set_memattr, moea_pvo_remove, moea_remove_write,
moea_protect, moea_pte_insert, moea_pte_spill, and
moea_mapdev_attr. SO the ptesync usage in 32-bit code
is fairly extensive.

ptesync seems to have an instruction encoding that is
reserved according to MPCFPE32B.pdf and MPC750UM.pdf .
eieio documented but not listed as context-synchronizing.
But it orders the tlbie vs. tlbsync: that is documented.
sync is not listed as context-synchronizing either. (This
matters later.)

_mtx_lock_spin_cookie returning does a bunch of memory
reads (at least from the stack) after any isync it might
execute. When prior code skips _mtx_lock_spin_cookie,
there is an isync involved.

The MPCFPE32B.pdf Data Access Synchronization table
indicates that for tlbie:

Required Prior: Context-synchronizing
Required After: Context-synchronizing or sync

It also has a note (not covering all the multiprocessor
system issues, as noted in the note after this one):

QUOTE
For data accesses, the context synchronizing instruction before the tlbie, or tlbia instruction ensures that all
memory accesses, due to preceding instructions, have completed to a point at which they have reported all interrupts
that may be caused. The context synchronizing instruction after the tlbie, or tlbia ensures that subsequent memory
accesses will not use the TLB entries being invalidated. It does not ensure that all memory accesses previously
translated by the TLB entries being invalidated have completed with respect to memory or, for tlbie or tlbia, that R
and C bit updates associated with those memory accesses have completed; if these completions must be ensured, the
tlbie, or tlbia must be followed by a sync instruction rather than by a context synchronizing instruction.
END QUOTE


So it looks to me like if the MPC750 (and, so, iMac G3)
is to be supported, there should (also?) be an isync
before the tlbie.

As for after the tlbie, the code is supposed to work
for multiprocessors so sync would appear to be required
by the note (to be sure tlbsync completed). But,
for the self-same (or only) processor, it would appear
that the "does not ensure that all memory accesses . . ."
material means that an isync is possibly also necessary.

It is not clear what the ptesync instruction encoding
leads to on the MPC750. Is it treated as MPC750 sync?
As a no-op? As something else? The MPCFPE32B.pdf and
MPC750UM.pdf indicate that sync is appropriate (with
the L=0 encoding specifically, in more modern terms).


Of course, allowing 32-bit FreeBSD to run on powerpc64
hardware may well not be able to use just the 32-bit's
MPCFPE32B.pdf and MPC750UM.pdf criteria related to tlbie.
This may well be more important than iMac G3's or other
MPC750 like contexts.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)