Re: STM32MP157

The root cause of zeroed stack in my setup with STM32MP157 seems to be in 
the TEX remapping registers values, calculated by pmap_set_tex(void). It 
can be a wrong assumption, as I'm not sure of any hardware (OTP/Security) 
tunings, that might be in effect....
Blindly copied values for PRRR & NMRR registers from Linux kernel (6.1.28), 
supplied by STM in their distribution I could avoid the trouble with zeroed 
stack memory. The hardcoded values are:
rg ".equ\s+(PR|NM)RR" arch/arm/mm/*
arch/arm/mm/proc-v7-3level.S
110:.equ PRRR, 0xeeaa4400 @ MAIR0
111:.equ NMRR, 0xff000004 @ MAIR1
arch/arm/mm/proc-v7-2level.S
137:.equ PRRR, 0xff0a81a8
138:.equ NMRR, 0x40e040e0
I (wrongly?) assume, 2level is related to the ones, calculated by 
pmap_set_tex.
Narrowing down the inconsistency among values found in Linux kernel and 
calculated here, I came to the result that the type of inner cache, tuned 
for first value of tex_classes lut ruins memory:
TEX(PRRR_MEM, NMRR_WB_WA, NMRR_WB_WA, 0), /* 0 - ATTR_WB_WA */ - zeroes 
stack mem on TLB flush
TEX(PRRR_MEM, NMRR_WB, NMRR_WB_WA, 0), /* 0 - ATTR_WB_WA */ - zeroes stack 
mem on TLB flush
TEX(PRRR_MEM, NMRR_WT, NMRR_WB_WA, 0), /* 0 - ATTR_WB_WA */ - DOES NOT 
zeroes stack mem on TLB flush
TEX(PRRR_MEM, NMRR_NC, NMRR_WB_WA, 0), /* 0 - ATTR_WB_WA */ - DOES NOT 
zeroes stack mem on TLB flush
Need further investigation....
Any clues?
Stan
Stanislav Silnicki wrote:
STM32MP15x port current progress:
... it took some time to solder down the probe and establish conveniences 
with remote kernel debug...
So far, I need to understand, whither or not I face an expected behavior?
It looks like tlb_flush_all_local() 
(https://github.com/freebsd/freebsd-src/blob/525ecfdad597980ea4cd59238e24c8530dbcd31d/sys/arm/arm/pmap-v6.c#L512C14-L512C14)

destroys stack memory, where return address is stored, so when stored lr is 
poped into pc, it is 0x0....
here is a dump of my debug session around that code:
508 cp15_prrr_set(prrr);
(kgdb) i f
Stack level 0, frame at 0xc0784598:
pc = 0xc0557b44 in pmap_set_tex (/usr/src/sys/arm/arm/pmap-v6.c:508); saved 
pc = 0xc055254c
called by frame at 0xc07847e0
source language c.
Arglist at 0xc0784590, args: 
Locals at 0xc0784590, Previous frame's sp is 0xc0784598
Saved registers:
r4 at 0xc0784580, r5 at 0xc0784584, r6 at 0xc0784588, r10 at 0xc078458c, 
r11 at 0xc0784590, lr at 0xc0784594
(kgdb) x 0xc0784594
0xc0784594: 0xc055254c
(kgdb) n
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x600001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
509 cp15_nmrr_set(nmrr);
(kgdb) x 0xc0784594
0xc0784594: 0xc055254c
(kgdb) n
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x600001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
512 tlb_flush_all_local();
(kgdb) x 0xc0784594
0xc0784594: 0xc055254c
(kgdb) si
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x600001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
_CP15_TLBIALL () at /usr/src/sys/arm/include/cpu-v6.h:147
147 _WF0(_CP15_TLBIALL, CP15_TLBIALL) /* Invalidate entire unified TLB */
(kgdb) si
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x600001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
tlb_flush_all_local () at /usr/src/sys/arm/include/cpu-v6.h:340
340 dsb();
(kgdb) x 0xc0784594
0xc0784594: 0x00000000
(kgdb) si
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x600001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
pmap_set_tex () at /usr/src/sys/arm/arm/pmap-v6.c:513
513 }
(kgdb) si
Polling target stm32mp15x.cm4 failed, trying to reexamine
Failed to read memory at 0xe000ed00
Examination failed, GDB will be halted. Polling again in 100ms
Program stopped.
pmap_set_tex () at /usr/src/sys/arm/arm/pmap-v6.c:513
513 }
(kgdb) 
The address, referenced from error message (0xe000ed00) is mapped by STM's 
address space to "DDR extension (CA7 only) or Debug" with debug assigned to 
Cortex-M4 coprocessor.
I'm not sure, that it is an unexpected behav. (it is my first attempt to 
port to armv7), so need an advice.
Any?
Stan
Stanislav Silnicki wrote:
my current progress of STM32MP1xx port attempt:
Basically, both options to boot ubldr and then kernel in SPL & TF-A modes 
of u-boot are supported.
Initially, when I stated from TF-A options (security supervision from arm's 
trusted firmware os) I stuck with 
this line: 
https://github.com/freebsd/freebsd-src/blob/501bdf3001190686bf55d9d333cb533858c2cf2f/sys/arm/arm/locore-v6.S#L371 
<https://github.com/freebsd/freebsd-src/blob/501bdf3001190686bf55d9d333cb533858c2cf2f/sys/arm/arm/locore-v6.S#L371> 

control does not seem to reach beyond that point. I tried to enable the led 
with this assembly, which works anywhere else:
/* prepare to use LED @ GPIOA 13*/
ldr r0, =0x50002014
ldr r1, =0xFFFFDFFF
ldr r2, [r0]
and r2, r1, r2
/* Enable caches. */
mrc CP15_SCTLR(r7)
orr r7, #CPU_CONTROL_DC_ENABLE
orr r7, #CPU_CONTROL_IC_ENABLE
orr r7, #CPU_CONTROL_BPRD_ENABLE
mcr CP15_SCTLR(r7)
DSB
/* turn on GPIO LED */
str r2, [r0]
Control passes if I disable data cache ORing instruction, but then hangs on 
setting of new TTB couple lines below...
I thought it was due to security access control from TF-A, but it behaves 
the same w/o.
Any clues or expertise from other platforms are highly appreciated.
Stan
Stanislav Silnicki wrote:
Hello, I need an advice on the intial addresses layout when booting the 
kernel.
As I understand, ubldr is the proper way to boot the kernel. 
What is the correct address to load it, given that I wand to keep default 
KERNVIRTADDR (=0xc0000000) intact?
The u-boot loads ubldr this way in my current setup:
// FreeBSD version of environment
env_set("baudrate", "115200");
env_set("console", "ttyS0,115200");
env_set("stderr", "serial");
env_set("stdin", "serial");
env_set("stdout", "serial");
env_set("autostart", "yes");
env_set("loaderdev", "mmc 1"); // fbsd's ubldr treats our second mmc at 
index 1
env_set("bootcmd", "fatload mmc 2 0xc0000000 ubldr && bootelf");
then ubldr leads the kernel from mmc and passes it the control:
Loading kernel...
/boot/kernel/kernel text=0x1b4 text=0x5b8530 text=0x17d91c data=0xa4a60 
data=0x0+0x20c000 0x4+0x88810+0x4+0xe9d98|
Loading configured modules...
can't find '/boot/entropy'
can't find '/etc/hostid'
Using DTB compiled into kernel.
Kernel entry at 0xc0400200...
Kernel args: (null)
Then I could correctly pass the init of MMU and get into translated mode 
only with setting KERNVIRTADDR = 0xc4000000. It looks like kernel and 
loader clash somehow. 
So, just need to understand what is best option to load ubldr to? Will it 
affect further routines?
Stan
Stanislav Silnicki wrote:
OK, I got the idea!
As I realize, there is a minor bug in dtc, which affects compilation of 
stm's originated DTBs. I think it is best to make a PR into 
https://github.com/davidchisnall/dtc, 
<https://github.com/davidchisnall/dtc,> which I'm discussing with repo 
owner already. Please tell me, that I need to post PR into FBSD source tree 
if it is a shorter way for my fix.
My current setup is based upon QBASE1 from Karo-Electronics, but there is 
no goal to support/debug all peripherals, only a subset, including USB, 
I2C, SDMMC, DSI (and GPU, if lucky).
https://www.karo-electronics.com/fileadmin/download/Datasheets/QSMP-QSBASE1-Evalkit.pdf 
<https://www.karo-electronics.com/fileadmin/download/Datasheets/QSMP-QSBASE1-Evalkit.pdf> 

The vendor (Karo) supports only their Yocto based Linux distro. I spent 
some time to prepare TF-A & Uboot, capable with booting from SD card (that 
is more robust approach, as I think). STM does not promote/support 
non-secure boot approach with SPL, they insist to use TF-A or OPTEE, so it 
is pretty cumbersome path, I had to pass. I think, it will be easier for me 
to prepare some sort of README to support customization of uboot for STM's 
port and dig it somewhere in SRC rather than ; ; ; ;try to post PR's in 
those repos... Not sure, anyway.
So far I'm struggling with uart and early_printf...
I'm mixing this activity with my current occupation, so I don't expect 
rapid progress.
Thank you for clarifications! I'll try to do my best!
Stan
Oskar Holmlund wrote:
Hi Stanislav,
Please resend your message and CC the arm mailing list. It might be 
interesting for someone in the future ( 
https://lists.freebsd.org/archives/freebsd-arm/ ) otherwise they all think 
you stopped working on the STM32 port.
You should always keep one terminal up n running % man 9 style :)
https://wiki.freebsd.org/Phabricator will give you some information.
Its good if you get to know the people active in the ARM area, just keep an 
eye on the mailinglist and you will notice some names. If you have the 
opportunity to join any of the conference (eurobsdcon.org, asiabsdcon.org, 
bsdcan.org) to get to know even more people.
Join the IRC, for example on efnet #bsdmips is a good channel for ARM 
stuff.
Start with small changes and you will get feedback. From there the 
experience will grow and you will get into it all.
Correct, the device-tree import is from the Linux kernel and there is no 
prior work for the STM32MP15 SoCs. I cant find anything about the 
STM32MP15x in Net or OpenBSD either.
Probably because the STM32MP15 is the first(?) application SoC from ST?
1) hum? Do you need that for the reviews? It should be in SRC
2) Target branch is probably main.
3) It depends, if your custom board is opensource and availble around the 
globe through mouser/farnell/.. I dont see any problem. Otherwise pick the 
board from ST that you used as a reference when you developed the custom 
board.
For example in my day job we have a custom board built around the 
octavosystems OSD3358 that can be found in pocketbeagle/beaglebone black 
wifi. So the code thats goes into the FreeBSD project are all tested on the 
beaglebone boards. The stuff we need for our custom board like the device 
tree is keept as local patches in our inhouse build process.
//Oskar
2023-10-28 14:25 skrev Stanislav Silnicki:
Hi Oskar!
 > can you point me some guidelines to help myself to fit development
process? I'm sure, there is mature dev. culture around FBSD and I'd be
happy to make my contribution coherently from the beginning.
 > So far I'm done with setup of my account at reviews (keys, 2FA, etc.)
 > As I understand, there is no considerable progress with STM32,
although I have noticed, there are some DTS imported into the project.
 > > Indeed, several major questions:
1. repo url?
2. tagret branch for patches for stm32 hw?
3. is it possible to target custom board from our project, or I have
to ensure support for all dev. boards, provided by STM?
 > Thank you,Stan
 > Oskar Holmlund wrote:
 >> 2023-10-27 22:33 skrev Stanislav Silnicki:
<block
quote class="gmail_quote" type="cite" style="margin:0 0 0 
.8ex;border-left:1px #ccc solid;padding-left:1ex">
 >>> Hello!
I'm porting onto the subject hardware. So far the progress is
modest,
while the system boots (without console although...)
One of major issues is hardcoded value inside locore-v6.S. Here
is my
relevant post:
 >> > 
https://community.st.com/t5/stm32-mpus-embedded-software/freebsd-port-for-mp157c/td-p/601438

[1]
What is the best way to proceed? Patch, vendor kernel build,
something
else?
Stan
 > Links:
------
 >> [1] >
 > 
https://community.st.com/t5/stm32-mpus-embedded-software/freebsd-port-for-mp157c/td-p/601438
 
>> Hi Stan,
 >> Upload your patch to reviews.freebsd.org
 >> Love to see your other patches for the STM32MP15x.
kquote>
</block
 >> //Oskar