Examples of early boot variability in handle_kernel_slb_spill happening on 2-socket/2-core-each G5 PowerMac11,2

[Mark Millard]

I've switched to keeping counts of handle_kernel_slb_spill
usage for my experiments with the slb-miss behavior:

#if defined(__powerpc64__) && defined(AIM)
unsigned int at_or_after_KVA_START_slb_spill_srr0_count= 0;
unsigned int from_DMAP_START_upto_KVA_START_slb_spill_srr0_count= 0;
unsigned int below_DMAP_START_slb_spill_srr0_count= 0;

unsigned int at_or_after_KVA_START_slb_spill_dar_count= 0;
unsigned int from_DMAP_START_upto_KVA_START_slb_spill_dar_count= 0;
unsigned int below_DMAP_START_slb_spill_dar_count= 0;
/* Handle kernel SLB faults -- runs in real mode, all seat belts off */
void
handle_kernel_slb_spill(int type, register_t dar, register_t srr0)
{
        struct slb *slbcache;
        uint64_t slbe, slbv;
        uint64_t esid, addr;
        int i;

if (EXC_ISE==type) {
        if (0xe000000000000000u<=srr0)
                at_or_after_KVA_START_slb_spill_srr0_count++;
        else if (0xc000000000000000u<=srr0)
                from_DMAP_START_upto_KVA_START_slb_spill_srr0_count++;
        else
                below_DMAP_START_slb_spill_srr0_count++;
} else {
        if (0xe000000000000000u<=dar)
                at_or_after_KVA_START_slb_spill_dar_count++;
        else if (0xc000000000000000u<=dar)
                from_DMAP_START_upto_KVA_START_slb_spill_dar_count++;
        else
                below_DMAP_START_slb_spill_dar_count++;
}
. . .

That is only designed to deal with the "only bsp" time frame.

The examples below are for my use of prefaulting
about half of the 64-1 kernel entries:

        i = 0;
        for (va = virtual_avail; va < virtual_end && i<(n_slbs-1)/2; va += SEGMENT_LENGTH, i++)
                moea64_bootstrap_slb_prefault(va, 0);

(Some of the numbers checked are from before this loop
as well.)

The boots were a mix of:

The original power-on
shutdown -r now
shutdown -p now then power-on


The summary of the pictures of the samplings
of the counts goes like . . .

The moea64_bootstrap_slb_prefault loop had examples of:
(counts from just-before vs. just-after checked
for increases to non-zero)

A) No handle_kernel_slb_spill use in the boot.
B) dar<DMAP_START happening once
C) srr0<DMAP_START happening once (distinct boot)

bs_remap_earlyboot had examples of:
(counts from just-before vs. just-after checked
for increases)

A) No handle_kernel_slb_spill use in the boot.
B) DMAP_START<=dar<KVA_START happening twice
C) DMAP_START<=dar<KVA_START happening once

dpcpu_init had examples of:
(counts from just-before vs. just-after checked
for increases)

A) No handle_kernel_slb_spill use in the boot.
B) KVA_START<=dar happening once

No other before-vs-after counts examined showed
any changes in the counts: just the 3 stages. The
last sampling was just-after the
numa_mem_regions(...) call.

There were fairly small number of boots tried.
For a few I failed to get a picture soon
enough and so could not see at watch stage
a count had gone to non-zero.

While there may be a bias, both -r reboots and
power-ons varied in their behavior for the counts.

This also make clear the following about the live
slb content on such a G5 (relative to
handle_kernel_slb_spill):

0) There can be examples from/of dar:  dar<DMAP_START
1) There can be examples from/of srr0: srr0<DMAP_START
2) There can be examples from/of dar:  DMA_START<=dar<KVA_START
3) There can be examples from/of dar:  KVA_START<=dar
                                 (beyond what I prefaulted)

But none of those 4 is guaranteed to happen.

Hopefully none of (0-2) can lead to problems. (3)
is at least similar to normal behavior for the KVA.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)