[GSOC] bhyve port on ARM - weekly status report

[Mihai Carabas]

Hi everyone,

Last week I fulfilled an important step: I've managed to execute an
> instruction in the VirtualMachine and returning back with the HYP expection
> caused by executing a "wfi" instruction (which is disabled by
> configuration). Than I've added two more operations: mov r0,1 and add r4,
> r0 , 4 and both are executing OK (tested even step-by-step in the debugger).
>
> Further, I've read the exception status registers (HSR, HIFAR, HDFAR,
> HPFAR) in hyp-mode just before returning to the host. I've read them all
> even some of them are in state unknown depending on the exception type. I
> will let the C code to verify this.
>
> I also had a design issue and I had to read these registers in "struct
> hypctx"  first and when returning in the host, copy them back to the
> "vm_exit" structure which was stored in "struct vm" (which isn't mapped in
> HYP-mode and even if I map it, I have to do a lot of index calculation to
> place the values in the right place - so I've decided to duplicate the
> info). So right now, when executing the WFI instruction, it causes a HYP
> exception which sets VM_EXITCODE_HYP which will determine the userspace
> utility to exit.
>
> An issue that need to be tackled is the reset state of the coprocessor
> registers. I need to take them one-by-one and set this reset state probably
> using some state-table approach (like KVM is doing).
>
> I've dropped the reset-state work. I've only set some of them to the
compulsory value at the virtual machine creation. If needed I will come
back to this issue.

After a discussion with Peter, the next step was implementing emulation for
memory access, basically to be able to emulate MMIO devices.

I've started by decoding the HSR (syndrome register) and put prints on each
exception type. Than I've interpreted that Data Abort Exception (DABT)
caused by a stage-2 translation to be able to decode the instruction. I've
saved the decoded instruction (read/write, the register number, the access
size [b,h,w]) and the fault address and sent it to bhyve userspace utility.
Here I've used the MMIO infrastructure taken from X86 to find an emulated
address and call the instruction emulation function
(vmm_emulate_instruction). Further I've implemented the instruction
emulation function in sys/arm/vmm/vmm_instruction_emul.c to get the
register value and call the memread/memwrite callbacks accordingly.

At this point I have a functional MMIO emulation. I've started modifying
the bvm_console with some conditional code for ARM to replace the inl/outl
calls with some memory acccesses. I've used the consport.c from X86 to
emulate the bvm_console. I've added some instructions to write some
characters to memory, and these are printed ok.

Further I've compiled a FreeBSD guest with bvm_console and ran it in bhyve.
Here are the first prints of a FreeBSD guest running on bhyve-arm [1] after
solving some tedious bugs (I was trashing some temporary registers which
were saving the SCTRL and I was disabling the MMU in the guest whenever I
was context-switching between host and guest). Further I'm continuing with
solving different guest exceptions this week.

Thank you,
Mihai

[1]
# bhyvearm -b test
initarm: console initialized
 arg1 kmdp = 0xc107a6d4
 boothowto = 0x00000000
 dtbp = 0xc0428708
 lastaddr1: 0xc108c000
loader passed (static) kenv:
 no env, null ptr
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2015 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #23 r288062:288705M: Fri Jul 24 04:57:12 EEST 2015
    root at freebsd64
:/root/obj/arm.armv6/root/soc2015/mihai/bhyve-on-arm-head/sys/
FVP_VE_CORTEX_A15x1_GUEST arm
FreeBSD clang version 3.6.1 (tags/RELEASE_361/final 237755) 20150525
subsystem 1000000
   0xc0328b60(0)... hyp_handle_exception:386 DABT from guest at address
c112c000
 without a stage-2 fault translation