32-bit powerpc vs. 64-bit FreeBDS powermac_smp_timebase_sync and mttb: sets upper part to zero vs. not

[Mark Millard]

[In exploring why there is the time problems that lead to
sleep-gets-stuck, I ran into the below. It seems odd but
I've not found it tied to the failures.]

Both 32-bit and 64-bit PowerPC have a 64-bit Time Base Register.
But how it is used is an odd mix above mttb(time) for how the TBR
register is set for 32-bit powerpc vs. powerpc64 FreeBSD. (32-bit
FreeBSD runs on some powerpc64 hardware, as well.)

32-bit powerpc FreeBSD uses types u_long and u_int in some places,
both 32-bits in that context:

typedef u_int timecounter_get_t(struct timecounter *)
powermac_smp_timebase_sync(platform_t plat, u_long tb, int ap)

That last even leads to the high 32-bits bits being set to zero
explicitly. (Shown later.)

64-bit powerpc does not do that last part: u_long is 64-bits
64-bit powerpc does truncate for timecounter_get_t: u_int is 32
bits for powerpc64.

The code:

static __inline void
mttb(u_quad_t time)
{

       mtspr(TBR_TBWL, 0);
       mtspr(TBR_TBWU, (uint32_t)(time >> 32));
       mtspr(TBR_TBWL, (uint32_t)(time & 0xffffffff));
}

is used in powermac_smp_timebase_sync (which is bound to
platform_smp_timebase_sync for the context):

static void
powermac_smp_timebase_sync(platform_t plat, u_long tb, int ap)
{
        
        mttb(tb);
}

32-bit powerpc: necessarily (time >> 32) == 0u because of the "u_long tb".
64-bit powerpc: not necessarily, u_long tb holds 64 bits.

The details for 32-bit powerpc FreeBSD are:

<powermac_smp_timebase_sync> stwu    r1,-32(r1)
<powermac_smp_timebase_sync+0x4> stw     r31,24(r1)
<powermac_smp_timebase_sync+0x8> mr      r31,r1
<powermac_smp_timebase_sync+0xc> li      r0,0
<powermac_smp_timebase_sync+0x10> mtsprg  4,r0
<powermac_smp_timebase_sync+0x14> li      r9,0
<powermac_smp_timebase_sync+0x18> mtsprg  5,r9
<powermac_smp_timebase_sync+0x1c> mtsprg  4,r4
<powermac_smp_timebase_sync+0x20> lwz     r11,0(r1)
<powermac_smp_timebase_sync+0x24> lwz     r31,-8(r11)
<powermac_smp_timebase_sync+0x28> mr      r1,r11
<powermac_smp_timebase_sync+0x2c> blr

which sets the upper 32 bits of the tbr to zero via
"mtsprg 5,r9", where previously "li r9,0".

But some parts of the code are u_quad_t based, with
platform_smp_timebase_sync use implicitly truncating:
(Note: mftb returns u_quad_t but timecounter_get_t
based access is truncated.)


volatile static u_quad_t ap_timebase;
. . .
        platform_smp_timebase_sync(ap_timebase, 1); SO: truncated on 32-bit powerpc FreeBSD only
. . .
        ap_timebase = mftb() + 10; SO: not truncating here.
. . .
        platform_smp_timebase_sync(ap_timebase, 0); SO: 32-bit powerpc FreeBSD only: truncating


        static u_quad_t timebase = 0;
. . .
                timebase = mftb(); SO: not truncating here.
. . .
        platform_smp_timebase_sync(timebase, 0); SO: 32-bit powerpc FreeBSD only: truncating


        u_quad_t        tb, ttb; NOTE: The code below is never-32-bit-truncating.
. . .
        tb = mftb();
        ttb = tb + howmany((uint64_t)n * 1000000, ps_per_tick);
        while (tb < ttb)
                tb = mftb();


I'm unclear on why ap_timebase and timebase above are
u_quad_t instead of u_long, given the intended use with
platform_smp_timebase_sync . Debugging ability to see
more?

Note: there is a:

/usr/src/sys/powerpc/aim/mp_cpudep.c:   platform_smp_timebase_sync(0, 1);

but for both 32-bit powerpc FreeBSD and powerpc64 FreeBSD that sets
the TBR to zero for all 64 bits. (Removal of this on at least 32-bit
powerpc FreeBSD causes problems with wider TBR disagreements across
cpus.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)