[GSOC] bhyve port on ARM - weekly status report

[Mihai Carabas]

Hi everyone,

I've loaded the boot-wrapper image into the FastModel emulator. I've also
> given some custom parameters to load the kernel binary (kernel.bin
> generated by the build system) at physical address 0XC0004000 (the same as
> the virtual address). I've modified the original physical address in order
> to make the VA==PA and have a match between the dissambled code shown by
> Model Debugger and the C code even if I had no MMU (to make debugging
> easier until the MMU gets activated). In order to be able to debug in Model
> Debugger, I've loaded the kernel.debug ELF file just for debug purposes (I
> have this option). Even if I did this trick, the ASM instructions didn't
> match the C code loaded by the Model Debugger. After manually dumping the
> kernel.debug ELF file I've seen that the text area is shifted with 0x180 at
> 0xC0004000, being different than the start of the kernel text area. Peter
> noticed that ldscript.arm has an offset added before the .text section ( .
> = KERNVIRTADDR + SIZEOF_HEADERS;). I've removed the SIZEOF_HEADERS and now
> the C code matched with the ASM one.
>
> Further I've been executing step by step until the MMU gets activated for
> the first time (init_mmu) in the locore-v6 with the primitive page table (a
> 64MB zone mapped from the begining of the kernel). The simulation was
> failing when activating the second time the MMU (after building the kernel
> page table using the pmap). This was caused by the chosen physical address
> C0008000. The locore-v6 code was alligning this address at 1MB boundary
> thus the kernel start became C0000000, instead of C0004000. The C0000000
> was passed to the arm_physmem_kernaddr variable with in turn was used for  KERNEL_P2V
> macro, thus creating invalid mappings. After modifying in the boot-wrapper
> the address where the kernel text is loaded at C0200000 the problem was
> solved (I've used the 4000 offset initially because this was configured by
> the boot-wrapper).
>
> Now I've managed to boot-up the FreeBSD kernel until the console init
> function in initarm (cninit function). Further I'm trying to build an
> early_putc function using the semihosting feature exposed by the emulator
> until I get the console up.
>
> This week I've implemented the early_putc using the semihosting feature
provided by FastModel platform [1] (making a svc with 0x12345 in r0 and
some codification in the other register would write to the STDOUT of the
emulator). This helped me a lot to debug the problems in this stage of
porting (most of the problems caused by invalid DTS parsing from FreeBSD,
as you can see bellow).

First of all I've changed the DTS to RTSM_CORTEX_A15x1, which is more
suitable for my FVP_CORTEX_A15x1 fast model (I think the RTSM was present
in an earlier version of Fast Model) [2]. I had a lot of trouble with the
DTS parsing, finding some bugs in the FreeBSD:
- if the parent has a multi-valued address cell, the most significant
4-bytes are lost due to the u_long type. A more detailed description can be
found in my post here [3]
-  an issue in regions_to_avail function: end variable is vm_paddr_t which
is typedef at uint32_t. The problem is when having start at 0x80000000 and
the size of the memory 0x80000000 (2G) than the end variable which is start
+ size becomes 0 (overflow)
- a bug in simplebus_alloc_resource function: the addresses of the host are
indexes from the ranges of the parent node in the DTS. It cannot simply be
added to an address [4]

These issues implies modification in a lot of places in the codebase so
I've modified the DTS so to contain what FreeBSD expects (removed the
multi-valued address-cells, remove all the range and leaving only the
devices range). Also I've added only 1G of RAM to pass the second issue.
These should be tackled in the future in a more generic way.

Next I had to modify a little the boot-wrapper to enable the co-processor
register access for VFP driver. I will soon push the boot-wrapper in the
repo too.

I also had to add the SP804 timer from the VERSATILEPB (the generic timer
in the DTS didn't had addresses and the parser complained about this).

Right now I've managed to finally get to the mount root part [5].

PS: This week I will try to create a tutorial on the wiki on how to run the
FreeBSD on FastModel FVP_VE-CortexA15x1.

Thank you,
Mihai

[1]
https://svnweb.freebsd.org/socsvn/soc2015/mihai/bhyve-on-arm-head/sys/arm/fvp_ve-cortex_a15x1/fvp_ve-cortex_a15x1_semihosting.S?view=markup
[2]
https://svnweb.freebsd.org/socsvn/soc2015/mihai/bhyve-on-arm-head/sys/boot/fdt/dts/arm/fvp_ve-cortex_a15x1.dts?view=markup
[3] https://lists.freebsd.org/pipermail/freebsd-arm/2015-June/011627.html
[4] https://lists.freebsd.org/pipermail/freebsd-arm/2015-June/011639.html
[5] Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:tank
        cd9660:/dev/acd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot>