RPi3 not using SMP?
Mark Millard
marklmi at yahoo.com
Sat Feb 8 05:59:04 UTC 2020
[A note on avoiding a bad interpretation of
my evidence.]
On 2020-Feb-7, at 20:12, Mark Millard <marklmi at yahoo.com> wrote:
> On 2020-Feb-7, at 19:10, Mark Millard <marklmi at yahoo.com> wrote:
>
>> [I now have some bounds on when PSCI_FN_VERSION
>> gets a 0 result from smc #0 instead of the correct
>> result: 2. Hopefully I can narrow the range more.]
>>
>> On 2020-Feb-7, at 18:46, Mark Millard <marklmi at yahoo.com> wrote:
>>
>>
>>
>>> On 2020-Feb-7, at 17:19, bob prohaska <fbsd at www.zefox.net> wrote:
>>>
>>>> For some weeks now an RPi3 running -current has seemed rather slow....
>>>>
>>>> On looking at the early part of the boot message the processor
>>>> attributes seem rather scant:
>>>>
>>>> ......
>>>> elease APs...APs not started
>>>> CPU 0: ARM Cortex-A53 r0p4 affinity: 0
>>>> Instruction Set Attributes 0 = <CRC32>
>>>> Instruction Set Attributes 1 = <>
>>>> Processor Features 0 = <AdvSIMD,FP,EL3 32,EL2 32,EL1 32,EL0 32>
>>>> Processor Features 1 = <>
>>>> Memory Model Features 0 = <TGran4,TGran64,SNSMem,BigEnd,16bit ASID,1TB PA>
>>>> Memory Model Features 1 = <8bit VMID>
>>>> Memory Model Features 2 = <32bit CCIDX,48bit VA>
>>>> Debug Features 0 = <2 CTX BKPTs,4 Watchpoints,6 Breakpoints,PMUv3,Debugv8>
>>>> Debug Features 1 = <>
>>>> Auxiliary Features 0 = <>
>>>> Auxiliary Features 1 = <>
>>>> CPU 1: (null) (null) r0p0 affinity: 0
>>>> CPU 2: (null) (null) r0p0 affinity: 0
>>>> CPU 3: (null) (null) r0p0 affinity: 0
>>>> ............
>>>> In a top window, STATE is reported as RUN, rather than the
>>>> former CPUn.
>>>>
>>>> Is a software switch now required to enable multiprocessing?
>>>>
>>>> Or, could it be related to the lines:
>>>> psci0: PSCI version number mismatched with DT
>>>> as pointed out by Mark M in reference to the cpu_reset failed
>>>> problem, which is still manifest?
>>>>
>>>> The kernel is at r357644.
>>>
>>> Head -r356767's kernel does not have this problem for RPi3/4 used as
>>> aarch64 FreeBSD.
>>>
>>> Head -r356776 and later all have this problem for both RPi3 and RPi4.
>>>
>>> Note: There are no head versions between those.
>>>
>>> The console log shows evidence of the problem much earlier.
>>> Instead of saying:
>>>
>>> psci0: <ARM Power State Co-ordination Interface Driver> on ofwbus0
>>> psci0: PSCI version 0.2 compatible
>>>
>>> (once) it says:
>>>
>>> psci0: <ARM Power State Co-ordination Interface Driver> on ofwbus0
>>> psci0: PSCI version number 0 mismatched with DT, default 2
>>> device_attach: psci0 attach returned 6
>>>
>>> (and those 3 lines repeat in various places) for which none of
>>> them show up for -r356767 .
>>>
>>> Without identifying and using PSCI, the extra cores will not
>>> start and the cpu(s) will not reset. (PSCI is the ARM interface
>>> for doing such things.)
>>>
>>> I've no clue why, but the version number it gets in my RPi4
>>> experiments is 0. My only guess is that at some point
>>> memory important to ARM's PSCI operation has been touched
>>> and is no longer valid for the PSCI operation.
>>
>> I've been doing some crude printf-based information
>> gathering for RPi4B boots (based on head -r357529 just
>> because it is convenient in my context) and I have
>> learned some about the staging of when PSCI_FN_VERSION
>> works vs. when it is no longer working.
>>
>> For the below text extraction from a boot . . .
>> Lines with: "PSCI_FN_VERSION 2" are working cases.
>> Lines with: "PSCI_FN_VERSION 0" are no-longer-working cases.
>>
>> . . .
>> FreeBSD clang version 9.0.1 (git at github.com:llvm/llvm-project.git c1a0a213378a458fbea1a5c77b315c7dce08fd05) (based on LLVM 9.0.1)
>> uma_startup1 start: PSCI_FN_VERSION 2
>> uma_startup1 end: PSCI_FN_VERSION 2
>> uma_startup2 start: PSCI_FN_VERSION 2
>> uma_startup2 end: PSCI_FN_VERSION 2
>> VT(efifb): resolution 1824x984
>> module firmware already present!
>> psci_fdt_get_callfn: method 'smc'
>> psci_fdt_get_callfn: arm_smccc_hvc '0xffff0000008663b0'
>> psci_fdt_get_callfn: arm_smccc_smc '0xffff0000008663c8'
>> psci_init: PSCI_FN_VERSION 0
>> Starting CPU 1 (1)
>> Starting CPU 2 (2)
>> Starting CPU 3 (3)
>> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
>> random: unblocking device.
>> uma_startup3 start: PSCI_FN_VERSION 0
>> uma_startup3 start: PSCI_FN_VERSION 0
>> . . .
>>
>> So sometime after uma_startup2 ends but before psci_init
>> sets up the normal use of arm_smccc_smc things are
>> messed up.
>>
>> My guess is that one or more of the kernel memory
>> allocations ended up getting memory used by ARM's
>> PSCI implementation and the content was invalidated
>> for ARM's PSCI purposes.
>>
>
> The transition from working to not is before the
> debug.verbose_sysinit=1 output turns in to
> symbolic notation:
>
> . . .
> subsystem 1800000
> mi_startup: PSCI_FN_VERSION 2
> 0xffff00000047e170(0)... done.
> mi_startup: PSCI_FN_VERSION 2
> 0xffff0000007bda18(0)... done.
> mi_startup: PSCI_FN_VERSION 2
> 0xffff0000004407e4(0)... done.
> mi_startup: PSCI_FN_VERSION 2
> 0xffff0000004400c0(0xffff000000a75890)... done.
> mi_startup: PSCI_FN_VERSION 2
> 0xffff0000004400c0(0xffff000000a76340)... done.
> . . .
> mi_startup: PSCI_FN_VERSION 2
> 0xffff0000004400c0(0xffff000000aca328)... done.
> mi_startup: PSCI_FN_VERSION 2
> 0xffff0000004400c0(0xffff000000aca378)... done.
> mi_startup: PSCI_FN_VERSION 0
> 0xffff0000004400c0(0xffff000000aca5f8)... done.
> . . .
> mi_startup: PSCI_FN_VERSION 0
> 0xffff0000004c62b0(0)... done.
> mi_startup: PSCI_FN_VERSION 0
> 0xffff00000049e93c(0)... done.
> mi_startup: PSCI_FN_VERSION 0
> linker_preload(0)... Preloaded elf kernel "/boot/kernel/kernel" at 0xffff000001568000.
> Preloaded elf module "/boot/kernel/umodem.ko" at 0xffff000001571020.
> Preloaded elf module "/boot/kernel/ucom.ko" at 0xffff000001571838.
> Preloaded boot_entropy_cache "/boot/entropy" at 0xffff000001572010.
> module firmware already present!
> done.
> subsystem 1ffffff
> mi_startup: PSCI_FN_VERSION 0
> ucom_init(0)... done.
> subsystem 2000000
> mi_startup: PSCI_FN_VERSION 0
> procs_show_all_add(0)... done.
> . . .
>
>
> It turns out that 0xffff0000004400c0 is: malloc_init .
>
> It turns out that 0xffff000000aca378 is for: M_IFMADDR :
>
> . . .
>
Do not take the above as an indication of a stable stage for the
change to happen. As my activities change the memory allocation
patterns and the memory layout some (and possibly other sources
of variability are involved), where in the boot sequence moves
around.
For example, while the above was before the output
turned to symbolic notation, that need not be the
case in general.
So it is just an example of where is possible and limits a
kind of activity that is sufficient for the change in status
to happen, since it is a fairly specific example context.
The way things move around, I'm not likely to come up with
a narrower type of activity spanning the status change.
I have no evidence on if the Excluded Memory Regions
are sufficient or respected.
For reference, I'd used
set debug.verbose_sysinit=1
boot -v
for the above example and my own printf's were
involved. And I based this testing on head
-r357529 instead of directly on -r356776 . The
kernel was a non-debug build (with symbols).
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
More information about the freebsd-arm
mailing list