[RFC] Add support for Xen ARM guest on FreeBSD
Julien Grall
julien.grall at linaro.org
Tue Jan 14 21:01:15 UTC 2014
Hello,
During the last couple of months I have been working on porting FreeBSD on
Xen on ARM.
It's my first big project on FreeBSD and I'm not sure if I have cc the right
MLs and people. If not, could somebody point me to the right email address?
With this patch series I'm able to boot FreeBSD with a single VCPU on every
32 bits platform supported by Xen.
The patch series is divided in 5 parts:
* #1-#12: Clean up and bug fix with some options
* #13-#17: Update the Xen interface to Xen 4.4 and fix compilation
* #18-#32: Update Xen code to support ARM guest
* #33-#38: Update ARM code to support Xen guest
* #39-#40: Add platform code to support ARM guest
I have pushed a branch on my git based on Royger's pvh work version 10:
git clone git://xenbits.xen.org/people/julieng/freebsd.git -b xen-arm
http://xenbits.xen.org/gitweb/?p=people/julieng/freebsd.git;a=shortlog;h=refs/heads/xen-arm
If needed, I can send the patch one by one the different mailing list.
This new support brings 2 open questions (for both Xen and FreeBSD community).
When a new guest is created, the toolstack will generated a device tree which
will contains:
- The amount of memory
- The description of the platform (gic, timer, hypervisor node)
- PSCI node for SMP bringup
Until now, Xen on ARM supported only Linux-based OS. When the support of
device tree was added in Xen for guest, we chose to use Linux device tree
bindings (gic, timer,...). It seems that FreeBSD choose a different way to
implement device tree:
- strictly respect ePAR (for interrupt-parent property)
- gic bindings different (only 1 interrupt cell)
I would like to come with a common device tree specification (bindings, ...)
across every operating system. I know the Linux community is working on having
a device tree bindings out of the kernel tree. Does FreeBSD community plan to
work with Linux community for this purpose?
As specified early, the device tree can vary accross Xen version and user input
(eg the memory). I have noticed few places (mainly the timer) where the IRQs
number are harcoded.
In the long-term, it would be nice to see FreeBSD booting out-of-box (eg the
device tree is directly provided by the board firmware). I plan to add support
for Device Tree loading via Linux Boot ABI, it the way that Xen use to boot a
new guest.
The second question is related to memory attribute for page table. The early
page table setup by FreeBSD are using Write-Through memory attribute which
result to a likely random (not every time the same place) crash before the
real page table are initialized.
Replacing Write-Through by Write-Back made FreeBSD boot correctly. Even today,
I have no idea if it's a requirement from Xen or a bug (either in Xen or
FreeBSD).
The code is taking its first faltering steps, therefore the TODO is quite big:
- Try the code on x86 architecture. I did lots of rework for the event
channels code and didn't even try to compile
- Clean up event channel code
- Clean up xen control code
- Add support for Device Tree loading via Linux Boot ABI
- Fixing crashing on userspace. Could be related to the patch
series "arm SMP on Cortex-A15"
- Add guest SMP support
- DOM0 support?
Any help, comments, questions are welcomed.
Sincerely yours,
============= Instruction to test FreeBSD on Xen on ARM ===========
Xen upstream tree doesn't fully support ELF loading for ARM. You will need to
recompile the tools with the following 2 patches:
- https://patches.linaro.org/22228/
- https://patches.linaro.org/22227/
To compile and boot Xen on your board, you can refer to the wiki page:
http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions
The instruction to compile FreeBSD for Xen on ARM:
$ truncate -s 512 xenvm.img
$ sudo mdconfig -f xenvm.img -u0
$ sudo newfs /dev/md0
$ sudo mount /dev/md0 /mnt
$ sudo make TARGET_ARCH=armv6 kernel-toolchain
$ sudo make TARGET_ARCH=armv6 KERNCONF=XENHVM buildkernel
$ sudo make TARGET_ARCH=armv6 buildworld
$ sudo make TARGET_ARCH=armv6 DESTDIR=/mnt installworld distribution
$ echo "/dev/xbd0 / ufs rw 1 1" > /mnt/etc/fstab
$ vi /mnt/etc/ttys (add the line 'xc0 "/usr/libexec/getty Pc" xterm on secure")
$ sudo umount /mnt
$ sudo mdconfig -d u0
Then you can copy the rootfs and the kernel to DOM 0 on your board.
To boot the a FreeBSD your will required the following configuration file
$ cat freebsd.xl
kernel="kernel"
memory=64
name="freebsd"
vcpus=1
autoballon="off"
disk=[ 'phy:/dev/loop0,xvda,w' ]
$ losetup /dev/loop0 xenvm.img
$ xl create freebsd.xl
$ xl console freebsd
Julien Grall (40):
xen/netfront: Don't need to include machine/intr_machdep.h
xen/blkfront: Don't need to include machine/intr_machdep.h
xen/control: Remove include machine/intr_machdep.h
xen: Define xen_intr_handle_upcall in common headers
xen: Remove duplicate features.h header in i386 arch
xen/hypercall: Allow HYPERVISOR_console_write to take a const string
xen/timer: Make xen timer optional
xen/console: Fix build when DDB is enabled
xen/console: clean up identify callback
xen/xenstore: xs_probe should return BUS_PROBE_NOWILDCARD
xen/control: xctlr_probe shoud return BUS_PROBE_NOWILDCARD
xen/hypervisor: Be sure to set __XEN_INTERFACE_VERSION__
xen/interface: Update interface to Xen 4.4 headers
xen: Bump __XEN_INTERFACE_VERSION__ to 4.3
xen/baloon: Fix compilation with Xen 4.4 headers
xen/netfront: Fix compilation with Xen 4.4 headers
xen/gnttab: Fix compilation with Xen 4.4 headers
xen/ballon: Use correct type for frame list
xen/netback: Fix printf format for xen_pfn_t
xen/netfront: Use the correct type for rx_pfn_array
xen/gnttab: Use the right type for the frames
xen/gnttab: Export resume_frames
xen/gnttab: Add a guard for xenpci functions
xen/netfront: Define PTE flags per architecture
xen/control: Implement suspend has panic for ARM
xen: Introduce xen_pmap
xen: move x86/xen/xenpv.c in dev/xen/xenpv.c
xen/xenpv: Only add isa for x86 architecture
xen/xenpv: Protect xenpci code by DEV_XENPCI
xen: move xen_intr.c to common code
xen/evtchn: Rework the event channels handler to make it generic
xen/console: handle console for HVM
arm/timer: Handle timer with no clock-frequency property
arm/timer: WORKAROUND The timer should support DT...
arm: Detect new revision for cortex A15
arm: add ELFNOTE macro
arm: Implement disable_intr
arm: Use Write-Back as memory attribute for early page table
dts: Add xenvm.dts
arm: Add xenhvm platform
sys/amd64/conf/GENERIC | 4 +-
sys/amd64/include/apicvar.h | 1 -
sys/amd64/include/xen/hypercall.h | 2 +-
sys/amd64/include/xen/xen-os.h | 2 +
sys/amd64/include/xen/xenpmap.h | 2 +-
sys/arm/arm/cpufunc.c | 2 +
sys/arm/arm/generic_timer.c | 12 +-
sys/arm/arm/locore.S | 4 +-
sys/arm/conf/XENHVM | 97 +++
sys/arm/include/armreg.h | 2 +
sys/arm/include/asmacros.h | 26 +
sys/arm/include/cpufunc.h | 6 +
sys/arm/include/xen/hypercall.h | 50 ++
sys/arm/include/xen/synch_bitops.h | 52 ++
sys/arm/include/xen/xen-os.h | 29 +
sys/arm/include/xen/xenfunc.h | 4 +
sys/arm/include/xen/xenvar.h | 14 +
sys/arm/xenhvm/elf.S | 12 +
sys/arm/xenhvm/files.xenhvm | 18 +
sys/arm/xenhvm/hypercall.S | 107 +++
sys/arm/xenhvm/xen-dt.c | 148 ++++
sys/arm/xenhvm/xenhvm_machdep.c | 132 ++++
sys/boot/fdt/dts/xenvm.dts | 77 ++
sys/conf/files | 4 +-
sys/conf/options | 3 +
sys/conf/options.arm | 1 +
sys/dev/xen/balloon/balloon.c | 6 +-
sys/dev/xen/blkfront/blkfront.c | 1 -
sys/dev/xen/console/console.c | 28 +-
sys/dev/xen/console/xencons_ring.c | 41 +-
sys/dev/xen/control/control.c | 19 +-
sys/dev/xen/netback/netback.c | 6 +-
sys/dev/xen/netfront/netfront.c | 15 +-
sys/dev/xen/xenpci/xenpci.c | 2 +-
sys/dev/xen/xenpv.c | 133 ++++
sys/i386/conf/GENERIC | 4 +-
sys/i386/conf/XEN | 1 +
sys/i386/include/apicvar.h | 1 -
sys/i386/include/xen/features.h | 22 -
sys/i386/include/xen/hypercall.h | 2 +-
sys/i386/include/xen/xen-os.h | 2 +
sys/i386/include/xen/xenvar.h | 2 +-
sys/i386/xen/xen_machdep.c | 2 +-
sys/x86/xen/xen_intr.c | 1241 -------------------------------
sys/x86/xen/xenpv.c | 128 ----
sys/xen/gnttab.c | 23 +-
sys/xen/hypervisor.h | 3 +-
sys/xen/interface/arch-arm.h | 259 +++++--
sys/xen/interface/arch-arm/hvm/save.h | 2 +-
sys/xen/interface/arch-x86/hvm/save.h | 25 +-
sys/xen/interface/arch-x86/xen-mca.h | 2 +-
sys/xen/interface/arch-x86/xen-x86_32.h | 2 +-
sys/xen/interface/arch-x86/xen-x86_64.h | 2 +-
sys/xen/interface/arch-x86/xen.h | 31 +-
sys/xen/interface/callback.h | 4 +-
sys/xen/interface/dom0_ops.h | 2 +-
sys/xen/interface/domctl.h | 65 +-
sys/xen/interface/elfnote.h | 10 +-
sys/xen/interface/event_channel.h | 5 +-
sys/xen/interface/features.h | 16 +-
sys/xen/interface/gcov.h | 115 +++
sys/xen/interface/grant_table.h | 8 +-
sys/xen/interface/hvm/hvm_xs_strings.h | 80 ++
sys/xen/interface/hvm/ioreq.h | 20 +-
sys/xen/interface/hvm/params.h | 14 +-
sys/xen/interface/hvm/pvdrivers.h | 47 ++
sys/xen/interface/hvm/save.h | 4 +-
sys/xen/interface/io/blkif.h | 104 ++-
sys/xen/interface/io/console.h | 2 +-
sys/xen/interface/io/fbif.h | 2 +-
sys/xen/interface/io/kbdif.h | 2 +-
sys/xen/interface/io/netif.h | 47 +-
sys/xen/interface/io/pciif.h | 3 +-
sys/xen/interface/io/protocols.h | 5 +-
sys/xen/interface/io/tpmif.h | 68 +-
sys/xen/interface/io/usbif.h | 1 -
sys/xen/interface/io/vscsiif.h | 42 +-
sys/xen/interface/io/xenbus.h | 20 +-
sys/xen/interface/io/xs_wire.h | 7 +-
sys/xen/interface/kexec.h | 7 +-
sys/xen/interface/mem_event.h | 4 +-
sys/xen/interface/memory.h | 80 +-
sys/xen/interface/nmi.h | 7 +-
sys/xen/interface/physdev.h | 33 +-
sys/xen/interface/platform.h | 27 +-
sys/xen/interface/sched.h | 87 ++-
sys/xen/interface/sysctl.h | 47 +-
sys/xen/interface/tmem.h | 2 +-
sys/xen/interface/trace.h | 93 ++-
sys/xen/interface/vcpu.h | 2 +-
sys/xen/interface/version.h | 6 +-
sys/xen/interface/xen-compat.h | 2 +-
sys/xen/interface/xen.h | 128 +++-
sys/xen/interface/xenoprof.h | 2 +-
sys/xen/interface/xsm/flask_op.h | 8 +
sys/xen/xen-os.h | 2 +-
sys/xen/xen_intr.c | 1072 ++++++++++++++++++++++++++
sys/xen/xen_intr.h | 8 +-
sys/xen/xenstore/xenstore.c | 4 +-
99 files changed, 3339 insertions(+), 1791 deletions(-)
create mode 100644 sys/arm/conf/XENHVM
create mode 100644 sys/arm/include/xen/hypercall.h
create mode 100644 sys/arm/include/xen/synch_bitops.h
create mode 100644 sys/arm/include/xen/xen-os.h
create mode 100644 sys/arm/include/xen/xenfunc.h
create mode 100644 sys/arm/include/xen/xenvar.h
create mode 100644 sys/arm/xenhvm/elf.S
create mode 100644 sys/arm/xenhvm/files.xenhvm
create mode 100644 sys/arm/xenhvm/hypercall.S
create mode 100644 sys/arm/xenhvm/xen-dt.c
create mode 100644 sys/arm/xenhvm/xenhvm_machdep.c
create mode 100644 sys/boot/fdt/dts/xenvm.dts
create mode 100644 sys/dev/xen/xenpv.c
delete mode 100644 sys/i386/include/xen/features.h
delete mode 100644 sys/x86/xen/xen_intr.c
delete mode 100644 sys/x86/xen/xenpv.c
create mode 100644 sys/xen/interface/gcov.h
create mode 100644 sys/xen/interface/hvm/hvm_xs_strings.h
create mode 100644 sys/xen/interface/hvm/pvdrivers.h
create mode 100644 sys/xen/xen_intr.c
--
1.8.0
More information about the freebsd-xen
mailing list