A investigative hack that makes (for example) head -r356529 boot and operate normally an RPi4B (finally!): protect all armstub8-gic.bin's loaded content from replacement by the kernel

[Robert Crowston]

Very nice, thank you!

> I put in code to add a reserved memory region
> spanning the 2 pages at the beginning of the
> address space. This is enough to span all the
> armstub8-gic.bin content (that is loaded to
> address 0x0 in my test context).

I had looked at this when I first getting the armstub to work with the Rpi4, and indeed I had no idea this was a problem again today.

In Tymoshenko's u-boot patch for the pi3, we modify the dtb in memory to reserve these pages, before u-boot communicates the modified dtb to the operating system.

I used the same idea (https://github.com/agherzan/u-boot/commit/7d1d1ce63c1fe50b451ef0c730e1cd870b5bd440) when I first got the rpi4 to boot. Does sysutils/u-boot-rpi4 do this as well?

> I had done other investigative work earlier to
> find for sure where armstub8-gic.bin was being
> loaded in my example context: address 0x0.

I am quite sure this is correct. We need the absolute addresses to be accurate when we spin up the other CPUs.

(Apologies if this was already known, the thread has been split and it is a little hard to follow.)

-- Rob

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, 13 February 2020 02:00, Mark Millard via freebsd-arm <freebsd-arm at freebsd.org> wrote:

> [head -r356529 was just handy in my context.
> I expect this works for head -r356776 and
> later and probably before the problem was
> exposed as well.]
>
> The technique here is a quick hack to give
> evidence for what correct, general code
> needs to do that is not being done ( given
> the head -r356776 changes exposing an
> assumption no longer true and the use
> of the FreeBSD specific armstub8-gic.bin
> in my test context but armstub8.bin in
> some other contexts).
>
> I put in code to add a reserved memory region
> spanning the 2 pages at the beginning of the
> address space. This is enough to span all the
> armstub8-gic.bin content (that is loaded to
> address 0x0 in my test context).
>
> Note the first listed memory region in:
>
> Excluded memory regions:
> 0x00000000 - 0x00001fff, 0 MB ( 2 pages) NoAlloc NoDump
> 0x00000000 - 0x00000fff, 0 MB ( 1 pages) NoAlloc
> . . .
>
> My hack added that 2 page range as if it was in
> memreserve in the live DTB. I already had the
> failover code for fdt_get_reserved_mem to call
> fdt_get_reserved_regions (to try for memreserve
> when /reserved-memory is not found).
>
> So here is what enabled the "boots and operates
> normally" status for the RPi4B with a -r356776
> or later version of head (yes, all 4 cores
> operating):
>
> svnlite diff /usr/src/sys/dev/fdt/fdt_common.c
>
> ===============================================
>
> Index: /usr/src/sys/dev/fdt/fdt_common.c
>
> =========================================
>
> --- /usr/src/sys/dev/fdt/fdt_common.c (revision 357529)
> +++ /usr/src/sys/dev/fdt/fdt_common.c (working copy)
> @@ -485,7 +485,18 @@
>
> tuples = res_len / tuple_size;
> reservep = (pcell_t *)&reserve;
> +#ifdef aarch64
>
> -   //HACK!!!
> -   // Reserve the first few pages, for example to
> -   // preserve armstub8-gic.bin or armstub.bin
> -   // content.
> -   mr[0].mr_start= 0;
> -   mr[0].mr_size= 2*4096;
> -   tuples++;
> -   for (i = 1; i < tuples; i++) {
>     +#else
>     for (i = 0; i < tuples; i++) {
>     +#endif
>
>     rv = fdt_data_to_res(reservep, addr_cells, size_cells,
>     (u_long *)&mr[i].mr_start, (u_long *)&mr[i].mr_size);
>     @@ -512,6 +523,11 @@
>
>     root = OF_finddevice("/reserved-memory");
>     if (root == -1) {
>
> -         // Fail over to checking for and handling memreserve,
>
>
> -         // such as for a RPi4B.
>
>
> -         if (0 == fdt_get_reserved_regions(reserved,mreserved))
>
>
> -         	return (0);
>
>
> -         return (ENXIO);
>
>
>     }
>
>     I had done other investigative work earlier to
>     find for sure where armstub8-gic.bin was being
>     loaded in my example context: address 0x0.
>
>     I'm not trying to imply that assuming that load
>     address is appropriate. I simply do not know how
>     general address 0x0 is.
>
>     I'm only trying to imply that the page range
>     that ends up containing the armstub8*.bin
>     content should be excluded from what the kernel
>     will use for allocations.
>
>     I'm not trying to imply that DTB memreserve
>     content should be the involved. The memreserve
>     related code just happened to be where I
>     choose to do the "exclude more" hack.
>
>     I expect that causing the pages holding
>     armstub*.bin content to be excluded from FreeBSD
>     writing to any of those pages would also enable
>     RPi3's and the like to boot and operate as aarch64
>     FreeBSD examples for head -r356776 (and later,
>     other than other bugs). It did for the RPi4B
>     4 GiByte test context I used.
>
>     ===
>     Mark Millard
>     marklmi at yahoo.com
>     ( dsl-only.net went
>     away in early 2018-Mar)
>
>
> freebsd-arm at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org"