A investigative hack that makes (for example) head -r356529 boot and operate normally an RPi4B (finally!): protect all armstub8-gic.bin's loaded content from replacement by the kernel

[Mark Millard]

On 2020-Feb-13, at 07:22, Kyle Evans <kevans at freebsd.org> wrote:

> On Thu, Feb 13, 2020 at 9:05 AM Ralf Wenk <iz-rpi03 at hs-karlsruhe.de> wrote:
>> 
>> On 2020-02-13 at 15:26 +0100 Ralf Wenk wrote:
>>> On 2020-02-13 at 7:49 -0600 Kyle Evans wrote:
>>>> On Thu, Feb 13, 2020 at 7:43 AM Ralf Wenk <iz-rpi03 at hs-karlsruhe.de> wrote:
>>>>> 
>>>>> On 2020-02-12 at 18:00 -0800 Mark Millard wrote via freebsd-arm:
>>>>>> [...]
>>>>>> 
>>>>>> # svnlite diff /usr/src/sys/dev/fdt/fdt_common.c
>>>>>> Index: /usr/src/sys/dev/fdt/fdt_common.c
>>>>>> ===================================================================
>>>>>> --- /usr/src/sys/dev/fdt/fdt_common.c (revision 357529)
>>>>>> +++ /usr/src/sys/dev/fdt/fdt_common.c (working copy)
>>>>>> @@ -485,7 +485,18 @@
>>>>>> 
>>>>>>      tuples = res_len / tuple_size;
>>>>>>      reservep = (pcell_t *)&reserve;
>>>>>> +#ifdef __aarch64__
>>>>>> +     //HACK!!!
>>>>>> +     // Reserve the first few pages, for example to
>>>>>> +     // preserve armstub8-gic.bin or armstub.bin
>>>>>> +     // content.
>>>>>> +     mr[0].mr_start= 0;
>>>>>> +     mr[0].mr_size= 2*4096;
>>>>>> +     tuples++;
>>>>>> +     for (i = 1; i < tuples; i++) {
>>>>>> +#else
>>>>>>      for (i = 0; i < tuples; i++) {
>>>>>> +#endif
>>>>>> 
>>>>>>              rv = fdt_data_to_res(reservep, addr_cells, size_cells,
>>>>>>                      (u_long *)&mr[i].mr_start, (u_long *)&mr[i].mr_size);
>>>>>> @@ -512,6 +523,11 @@
>>>>>> 
>>>>>>      root = OF_finddevice("/reserved-memory");
>>>>>>      if (root == -1) {
>>>>>> +             // Fail over to checking for and handling memreserve,
>>>>>> +             // such as for a RPi4B.
>>>>>> +             if (0 == fdt_get_reserved_regions(reserved,mreserved))
>>>>>> +                     return (0);
>>>>>> +
>>>>>>              return (ENXIO);
>>>>>>      }
>>>>>> 
>>>>> 
>>>>> I can confirm that with your patch(es) my RPi3 does not freeze any more
>>>>> when loading mac_ntpd.ko. The patches are applied against r357853M.
>> 
>> An reboot is working again too.
>> 
>>>> Have you tested the RPi3 with just this second hunk of patch to
>>>> fallover to memreserve, or is the first hunk definitely required as
>>>> well?
>>> 
>>> Good question. I tested both hunks together.
>>> Will try what happens when just applying the second and report back.
>> 
>> Here it is:
>> Without the first hunk the system freezes again when loading mac_ntpd.ko.
>> Also the CPU information during boot for CPUs 1 to 3 looks strange again.
>> 
> 
> Yeah- I see it now; both armstubs are about 5k. I've raised an
> issue[0] with upstream for armstub/rpi bits to work out the proper
> solution, because I don't necessarily want to commit the workaround.
> I'll throw up the second hunk on phabricator for review by #arm/#arm64
> folks, because that seems to me the proper fallback.
> 
> I also discovered some issues when trying to read /memreserve/ with
> our dtc and filed a PR[1] to fix those.
> 
> Thanks,
> 
> Kyle Evans
> 
> [0] https://github.com/raspberrypi/tools/issues/107
> [1] https://github.com/davidchisnall/dtc/pull/59

The DTB information below is from:

U-Boot> fdt addr 0x7ef2000 
U-Boot> fdt print /       

on a RPi4B 4 GiByte.

On at least the RPi4B memreserve is not what causes
the first page to be excluded:

        memreserve = <0x3b400000 0x04c00000>;

Nor is memory at 0 the cause:

        memory at 0 {
                device_type = "memory";
                reg = <0x00000000 0x00000000 0x3b400000 0x00000000 0x40000000 0xbc000000>;
        };

(That also skips the memreserve area.)

I do not find anything in the DTB that indicates
to exclude the first page.

My hypothesis is that the FreeBSD code excludes
the page based on some less obvious relationship
that I'm not identifying.

There is the cpu-rlease-addr information that seems
to refer to some 1st memory page content:

        cpus {
                #address-cells = <0x00000001>;
                #size-cells = <0x00000000>;
                enable-method = "brcm,bcm2836-smp";
                phandle = <0x000000be>;
                cpu at 0 {
                        device_type = "cpu";
                        compatible = "arm,cortex-a72";
                        reg = <0x00000000>;
                        enable-method = "spin-table";
                        cpu-release-addr = <0x00000000 0x000000d8>;
                        phandle = <0x0000001d>;
                };
                cpu at 1 {
                        device_type = "cpu";
                        compatible = "arm,cortex-a72";
                        reg = <0x00000001>;
                        enable-method = "spin-table";
                        cpu-release-addr = <0x00000000 0x000000e0>;
                        phandle = <0x0000001e>;
                };
                cpu at 2 {
                        device_type = "cpu";
                        compatible = "arm,cortex-a72";
                        reg = <0x00000002>;
                        enable-method = "spin-table";
                        cpu-release-addr = <0x00000000 0x000000e8>;
                        phandle = <0x0000001f>;
                };
                cpu at 3 {
                        device_type = "cpu";
                        compatible = "arm,cortex-a72";
                        reg = <0x00000003>;
                        enable-method = "spin-table";
                        cpu-release-addr = <0x00000000 0x000000f0>;
                        phandle = <0x00000020>;
                };
        };




===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)