A investigative hack that makes (for example) head -r356529 boot and operate normally an RPi4B (finally!): protect all armstub8-gic.bin's loaded content from replacement by the kernel
Robert Crowston
crowston at protonmail.com
Sat Feb 15 10:36:46 UTC 2020
> So it may be sysutils/u-boot-rpi{3,4} that needs to
> arrange sufficient room to prevent messing up such.
> (Unless armstub8*.bin can adjust something that
> u-boot's EFI interface is based on for that initial
> "Reserved" area?)
The area to be reserved is already passed in register x1 by armstub to u-boot here:
https://github.com/gonzoua/rpi3-psci-monitor/blob/master/pscimon.S#L178
u-boot can read this register before it does anything else in save_boot_params in lowlevel_init.S. (e.g., https://github.com/RobCrowston/u-boot/blame/7d1d1ce63c1fe50b451ef0c730e1cd870b5bd440/board/raspberrypi/rpi/lowlevel_init.S#L38).
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, 13 February 2020 18:27, Mark Millard via freebsd-arm <freebsd-arm at freebsd.org> wrote:
> On 2020-Feb-13, at 09:55, Mark Millard <marklmi at yahoo.com> wrote:
>
> On 2020-Feb-13, at 09:36, Mark Millard <marklmi atyahoo.com> wrote:
>
> > > On 2020-Feb-13, at 08:50, Mark Millard <marklmi at yahoo.com> wrote:
> > >
> > > > On 2020-Feb-13, at 07:22, Kyle Evans <kevans at freebsd.org> wrote:
> > > >
> > > > > On Thu, Feb 13, 2020 at 9:05 AM Ralf Wenk iz-rpi03 at hs-karlsruhe.de wrote:
> > > > >
> > > > > > On 2020-02-13 at 15:26 +0100 Ralf Wenk wrote:
> > > > > >
> > > > > > > On 2020-02-13 at 7:49 -0600 Kyle Evans wrote:
> > > > > > >
> > > > > > > > On Thu, Feb 13, 2020 at 7:43 AM Ralf Wenk iz-rpi03 at hs-karlsruhe.de wrote:
> > > > > > > >
> > > > > > > > > On 2020-02-12 at 18:00 -0800 Mark Millard wrote via freebsd-arm:
> > > > > > > > >
> > > > > > > > > > [...]
> > > > > > > > > >
> > > > > > > > > > svnlite diff /usr/src/sys/dev/fdt/fdt_common.c
> > > > > > > > > >
> > > > > > > > > > ===============================================
> > > > > > > > > >
> > > > > > > > > > Index: /usr/src/sys/dev/fdt/fdt_common.c
> > > > > > > > > >
> > > > > > > > > > =========================================
> > > > > > > > > >
> > > > > > > > > > --- /usr/src/sys/dev/fdt/fdt_common.c (revision 357529)
> > > > > > > > > > +++ /usr/src/sys/dev/fdt/fdt_common.c (working copy)
> > > > > > > > > > @@ -485,7 +485,18 @@
> > > > > > > > > > tuples = res_len / tuple_size;
> > > > > > > > > > reservep = (pcell_t *)&reserve;
> > > > > > > > > > +#ifdef aarch64
> > > > > > > > > >
> > > > > > > > > > - //HACK!!!
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - // Reserve the first few pages, for example to
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - // preserve armstub8-gic.bin or armstub.bin
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - // content.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - mr[0].mr_start= 0;
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - mr[0].mr_size= 2*4096;
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - tuples++;
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - for (i = 1; i < tuples; i++) {
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > +#else
> > > > > > > > > > for (i = 0; i < tuples; i++) {
> > > > > > > > > > +#endif
> > > > > > > > > >
> > > > > > > > > > rv = fdt_data_to_res(reservep, addr_cells, size_cells,
> > > > > > > > > > (u_long *)&mr[i].mr_start, (u_long *)&mr[i].mr_size);
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > @@ -512,6 +523,11 @@
> > > > > > > > > > root = OF_finddevice("/reserved-memory");
> > > > > > > > > > if (root == -1) {
> > > > > > > > > >
> > > > > > > > > > - // Fail over to checking for and handling memreserve,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - // such as for a RPi4B.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - if (0 == fdt_get_reserved_regions(reserved,mreserved))
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - return (0);
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - return (ENXIO);
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I can confirm that with your patch(es) my RPi3 does not freeze any more
> > > > > > > > > when loading mac_ntpd.ko. The patches are applied against r357853M.
> > > > > >
> > > > > > An reboot is working again too.
> > > > > >
> > > > > > > > Have you tested the RPi3 with just this second hunk of patch to
> > > > > > > > fallover to memreserve, or is the first hunk definitely required as
> > > > > > > > well?
> > > > > > >
> > > > > > > Good question. I tested both hunks together.
> > > > > > > Will try what happens when just applying the second and report back.
> > > > > >
> > > > > > Here it is:
> > > > > > Without the first hunk the system freezes again when loading mac_ntpd.ko.
> > > > > > Also the CPU information during boot for CPUs 1 to 3 looks strange again.
> > > > >
> > > > > Yeah- I see it now; both armstubs are about 5k. I've raised an
> > > > > issue[0] with upstream for armstub/rpi bits to work out the proper
> > > > > solution, because I don't necessarily want to commit the workaround.
> > > > > I'll throw up the second hunk on phabricator for review by #arm/#arm64
> > > > > folks, because that seems to me the proper fallback.
> > > > > I also discovered some issues when trying to read /memreserve/ with
> > > > > our dtc and filed a PR[1] to fix those.
> > > > > Thanks,
> > > > > Kyle Evans
> > > > > [0] https://github.com/raspberrypi/tools/issues/107
> > > > > [1] https://github.com/davidchisnall/dtc/pull/59
> > > >
> > > > The DTB information below is from:
> > > > U-Boot> fdt addr 0x7ef2000
> > > > U-Boot> fdt print /
> > > > on a RPi4B 4 GiByte.
> > > > On at least the RPi4B memreserve is not what causes
> > > > the first page to be excluded:
> > > >
> > > > memreserve = <0x3b400000 0x04c00000>;
> > > >
> > > >
> > > > Nor is memory at 0 the cause:
> > > >
> > > > memory at 0 {
> > > > device_type = "memory";
> > > > reg = <0x00000000 0x00000000 0x3b400000 0x00000000 0x40000000 0xbc000000>;
> > > > };
> > > >
> > > >
> > > > (That also skips the memreserve area.)
> > > > I do not find anything in the DTB that indicates
> > > > to exclude the first page.
> > > > My hypothesis is that the FreeBSD code excludes
> > > > the page based on some less obvious relationship
> > > > that I'm not identifying.
> > > > There is the cpu-rlease-addr information that seems
> > > > to refer to some 1st memory page content:
> > > >
> > > > cpus {
> > > > #address-cells = <0x00000001>;
> > > > #size-cells = <0x00000000>;
> > > > enable-method = "brcm,bcm2836-smp";
> > > > phandle = <0x000000be>;
> > > > cpu at 0 {
> > > > device_type = "cpu";
> > > > compatible = "arm,cortex-a72";
> > > > reg = <0x00000000>;
> > > > enable-method = "spin-table";
> > > > cpu-release-addr = <0x00000000 0x000000d8>;
> > > > phandle = <0x0000001d>;
> > > > };
> > > > cpu at 1 {
> > > > device_type = "cpu";
> > > > compatible = "arm,cortex-a72";
> > > > reg = <0x00000001>;
> > > > enable-method = "spin-table";
> > > > cpu-release-addr = <0x00000000 0x000000e0>;
> > > > phandle = <0x0000001e>;
> > > > };
> > > > cpu at 2 {
> > > > device_type = "cpu";
> > > > compatible = "arm,cortex-a72";
> > > > reg = <0x00000002>;
> > > > enable-method = "spin-table";
> > > > cpu-release-addr = <0x00000000 0x000000e8>;
> > > > phandle = <0x0000001f>;
> > > > };
> > > > cpu at 3 {
> > > > device_type = "cpu";
> > > > compatible = "arm,cortex-a72";
> > > > reg = <0x00000003>;
> > > > enable-method = "spin-table";
> > > > cpu-release-addr = <0x00000000 0x000000f0>;
> > > > phandle = <0x00000020>;
> > > > };
> > > > };
> > > >
> > >
> > > Looking at the code there is:
> > >
> > > /* Load the physical memory ranges */
> > > efihdr = (struct efi_map_header *)preload_search_info(kmdp,
> > > MODINFO_METADATA | MODINFOMD_EFI_MAP);
> > > if (efihdr != NULL)
> > > add_efi_map_entries(efihdr);
> > >
> > >
> > > #ifdef FDT
> > > else {
> > > /* Grab physical memory regions information from device tree. */
> > > if (fdt_get_mem_regions(mem_regions, &mem_regions_sz,
> > > NULL) != 0)
> > > panic("Cannot get physical memory regions");
> > > arm_physmem_hardware_regions(mem_regions, mem_regions_sz);
> > > }
> > > if (fdt_get_reserved_mem(mem_regions, &mem_regions_sz) == 0)
> > > arm_physmem_exclude_regions(mem_regions, mem_regions_sz,
> > > EXFLAG_NODUMP | EXFLAG_NOALLOC);
> > > #endif
> > >
> > > /* Exclude the EFI framebuffer from our view of physical memory. */
> > > efifb = (struct efi_fb *)preload_search_info(kmdp,
> > > MODINFO_METADATA | MODINFOMD_EFI_FB);
> > > if (efifb != NULL)
> > > arm_physmem_exclude_region(efifb->fb_addr, efifb->fb_size,
> > > EXFLAG_NOALLOC);
> > >
> > >
> > > . . .
> > > if (boothowto & RB_VERBOSE) {
> > > print_efi_map_entries(efihdr);
> > > arm_physmem_print_tables();
> > > }
> > > It looks to me like the boot -v text:
> > >
> > > Type Physical Virtual #Pages Attr
> > > Reserved 000000000000 0 00000001 WB
> > >
> > >
> > > ConventionalMemory 000000001000 1000 00007ef1 WB
> > > BootServicesData 000007ef2000 7ef2000 0000001c WB
> > > ConventionalMemory 000007f0e000 7f0e000 00029f93 WB
> > > BootServicesData 000031ea1000 31ea1000 00000001 WB
> > > LoaderData 000031ea2000 31ea2000 00008001 WB
> > > LoaderCode 000039ea3000 39ea3000 000000a6 WB
> > > Reserved 000039f49000 39f49000 00000007 WB
> > > BootServicesData 000039f50000 39f50000 00000001 WB
> > > Reserved 000039f51000 39f51000 00000002 WB
> > > RuntimeServicesData 000039f53000 39f53000 00000001 WB RUNTIME
> > > Reserved 000039f54000 39f54000 00000001 WB
> > > BootServicesData 000039f55000 39f55000 00000002 WB
> > > RuntimeServicesData 000039f57000 39f57000 00000001 WB RUNTIME
> > > LoaderData 000039f58000 39f58000 00001408 WB
> > > RuntimeServicesCode 00003b360000 3b360000 00000010 WB RUNTIME
> > > LoaderData 00003b370000 3b370000 00000090 WB
> > > BootServicesData 000040000000 40000000 000bc000 WB
> > > MemoryMappedIO 0000fe100000 fe100000 00000001 RUNTIME
> > > is from print_efi_map_entries via the efihdr instead
> > > of from the FreeBSD FDT code and the DTB.
> > > So is it u-boot that provides the efihdr for which
> > > add_efi_map_entries generated those regions?
> > > That might explain why I do not find matching DTB
> > > material for all of it.
> >
> > Looks like the efi memory map traces back to the loader
> > and its use of GetMemoryMap (as far as FreeBSD goes):
> >
> > grep -r "GetMemoryMap" /usr/src/sys/ | more
> >
> > ============================================
> >
> > /usr/src/sys/amd64/amd64/machdep.c: * Memory map data provided by UEFI via the GetMemoryMap
> > /usr/src/sys/arm64/arm64/machdep.c: * Memory map data provided by UEFI via the GetMemoryMap
> > /usr/src/sys/arm/arm/machdep_boot.c: * Memory map data provided by UEFI via the GetMemoryMap
> > /usr/src/sys/contrib/edk2/Include/Uefi/UefiSpec.h: EFI_GET_MEMORY_MAP GetMemoryMap;
> >
> > grep -r "GetMemoryMap" /usr/src/stand/ | more
> >
> > ==============================================
> >
> > /usr/src/stand/efi/loader/copy.c: status = BS->GetMemoryMap(&sz, map, &key, &dsz, &dver);
> > /usr/src/stand/efi/loader/main.c: status = BS->GetMemoryMap(&sz, 0, &key, &dsz, &dver);
> > /usr/src/stand/efi/loader/main.c: status = BS->GetMemoryMap(&sz, map, &key, &dsz, &dver);
> > /usr/src/stand/efi/loader/bootinfo.c: status = BS->GetMemoryMap(&sz, mm, &efi_mapkey, &dsz, &mmver);
> > /usr/src/stand/efi/loader/bootinfo.c: printf("%s: GetMemoryMap error %lu\n", func,
> > /usr/src/stand/efi/include/efiapi.h: EFI_GET_MEMORY_MAP GetMemoryMap;
>
> Here is the memmap from the loader that is just based on EFI's
> GetMemoryMap from what I can tell from a wuick look:
>
> Hit [Enter] to boot immediately, or any other key for command prompt.
> Booting [/boot/kernel/kernel] in 8 seconds...
>
> Type '?' for a list of commands, 'help' for more detailed help.
> OK memmap
> Type Physical Virtual #Pages Attr
> Reserved 000000000000 000000000000 00000001 WB
> ConventionalMemory 000000001000 000000001000 00007ef1 WB
> BootServicesData 000007ef2000 000007ef2000 0000001c WB
> ConventionalMemory 000007f0e000 000007f0e000 00029f93 WB
> BootServicesData 000031ea1000 000031ea1000 00000001 WB
> LoaderData 000031ea2000 000031ea2000 00004000 WB
> BootServicesData 000035ea2000 000035ea2000 00000001 WB
> LoaderData 000035ea3000 000035ea3000 00004000 WB
> LoaderCode 000039ea3000 000039ea3000 000000a6 WB
> Reserved 000039f49000 000039f49000 00000007 WB
> BootServicesData 000039f50000 000039f50000 00000001 WB
> Reserved 000039f51000 000039f51000 00000002 WB
> RuntimeServicesData 000039f53000 000039f53000 00000001 WB
> Reserved 000039f54000 000039f54000 00000001 WB
> BootServicesData 000039f55000 000039f55000 00000002 WB
> RuntimeServicesData 000039f57000 000039f57000 00000001 WB
> LoaderData 000039f58000 000039f58000 00001408 WB
> RuntimeServicesCode 00003b360000 00003b360000 00000010 WB
> LoaderData 00003b370000 00003b370000 00000090 WB
> BootServicesData 000040000000 000040000000 000bc000 WB
> MemoryMappedIO 0000fe100000 0000fe100000 00000001
> OK
>
> So at the loader time frame, the 2nd page of memory is not
> Reserved (in EFI terms).
>
> It would appear to me that avoiding messing up the
> armstub8*.bin content should apply this early as well.
>
> So it may be sysutils/u-boot-rpi{3,4} that needs to
> arrange sufficient room to prevent messing up such.
> (Unless armstub8*.bin can adjust something that
> u-boot's EFI interface is based on for that initial
> "Reserved" area?)
>
>
> ========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
>
> Mark Millard
> marklmi at yahoo.com
> ( dsl-only.net went
> away in early 2018-Mar)
>
> freebsd-arm at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org"
More information about the freebsd-arm
mailing list