Linux on BHyVe in 10.0-RELEASE

Aryeh Friedman aryeh.friedman at gmail.com
Thu Jan 23 22:26:23 UTC 2014 

I will have this or something like this encapslated into the petitecloud
run scripts by monday (as well fixing the issues in the other thread)


On Thu, Jan 23, 2014 at 5:20 PM, Jonas Bülow <jonas.bulow at gmail.com> wrote:

> Hi,
>
> I managed to get it working again. I did the last grub step wrong. So, the
> step-by-step instructions how to install ubuntu 13.10 are:
>
> kldload vmm
> kldload if_tap
> kldload bridgestp
> kldload if_bridge
> ifconfig tap0 create
> ifconfig bridge0 create
> ifconfig bridge0 up
> ifconfig bridge0 addm tap0 addm em0
>
> mkdir ubuntu
> ...fetch ubuntu13.10 iso image and put it in ubuntu/ubuntu.iso
> create the file ubuntu/device.map with the following content (2 lines):
> (hd0) ./ubuntu/ubuntu.img
> (cd0) ./ubuntu/ubuntu.iso
>
> truncate -s 8G ubuntu/ubuntu.img
>
> grub-bhyve -r cd0 -m ./ubuntu/device.map -M 2048 ubuntu
> /usr/sbin/bhyve -c 1 -m 2048 -A -I -H -s 0,hostbridge -s
> 2,virtio-blk,./ubuntu/ubuntu.img -s 3,virtio-net,tap0 -s
> 4,ahci-cd,./ubuntu/ubuntu.iso -S 31,uart,stdio ubuntu
>
> ...install ubuntu.
>
> bhyvectl --destroy --vm=ubuntu
>
> grub-bhyve -r hd0,msdos1 -m ./ubuntu/device.map -M 2048 ubuntu
> ...select ubuntu
> ...see some errors:
>
> error: file `/boot/grub/x86_64-emu/all_video.mod' not found.
>
> error: file `/boot/grub/x86_64-emu/gzio.mod' not found.
>
> error: file `/boot/grub/x86_64-emu/part_msdos.mod' not found.
>
> error: file `/boot/grub/x86_64-emu/ext2.mod' not found.
>
> /usr/sbin/bhyve -c 1 -m 2048 -A -I -H -s 0,hostbridge -s
> 2,virtio-blk,./ubuntu/ubuntu.img -s 3,virtio-net,tap0 -s
> 4,ahci-cd,./ubuntu/ubuntu.iso -S 31,uart,stdio ubuntu
> ...boots
> ...login
>
> jonas at ubuntu:~$ lsb_release  -a
>
> No LSB modules are available.
>
> Distributor ID: Ubuntu
>
> Description: Ubuntu 13.10
>
> Release: 13.10
>
> Codename: saucy
>
> jonas at ubuntu:~$ uname -a
>
> Linux ubuntu 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013
> x86_64 x86_64 x86_64 GNU/Linux
>
>
> Note: there is no clean way of exiting a bhyve vm other than killing it
> (after halt or shutdown in the guest). The vm must be destroyed before
> starting it again. Trying to run the same bhyve command line again just
> hangs (vmidle, high CPU). procstat gives:
>
>   PID    TID COMM             TDNAME           KSTACK
>
>  2381 100531 bhyve            mevent           <running>
>
>  2381 100532 bhyve            vtnet-3:0 tx     mi_switch
> sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait
> __umtx_op_wait_uint_private amd64_syscall Xfast_syscall
>
>  2381 100533 bhyve            vcpu 0           mi_switch sleepq_timedwait
> msleep_spin_sbt vm_run vmmdev_ioctl devfs_ioctl_f kern_ioctl sys_ioctl
> amd64_syscall Xfast_syscall
>
>
>
> /J
>
>
>
>
> On Thu, Jan 23, 2014 at 10:45 PM, Neel Natu <neelnatu at gmail.com> wrote:
>
> > Hi Jonas,
> >
> > On Thu, Jan 23, 2014 at 1:02 PM, Jonas Bülow <jonas.bulow at gmail.com>
> > wrote:
> > > I had one working ubuntu13.10 vm instance but I'm unable to reproduce
> it.
> > > When I try to start the vm I get:
> > >
> > > /usr/sbin/bhyve -c 1 -m 2048 -A -I -H -s 0,hostbridge -s
> > > 2,virtio-blk,./vm/vm1/vm1.img -s 3,virtio-net,tap1 -s
> > > 4,ahci-cd,./vm/vm1/vm1.iso -S 31,uart,stdio vm1
> > >
> > > vm exit[0]
> > >
> > >           reason VMX
> > >
> > >                                    rip 0x0000000000000000
> > >
> > >
> > > inst_length 0
> > >
> > >
> > >                       error 0
> > >
> > >
> > >                                               exit_reason 33
> > >
> > >
> > >
> > > qualification 0x0000000000000000
> > >
> >
> > Looks like you are running into every possible problem there is :-)
> >
> > Anyways, could you confirm that you are loading the guest kernel into
> > 'vm1' using grub-bhyve before executing the command above?
> >
> > best
> > Neel
> >
> > >
> > >
> > >
> > >
> > > /J
> > >
> > >
> > > On Thu, Jan 23, 2014 at 5:00 PM, Jonas Bülow <jonas.bulow at gmail.com>
> > wrote:
> > >
> > >> I will describe the steps when I manage to reproduce the installation.
> > >> Right now I get stuck with undestroyable vm instances:
> > >>
> > >> root at bulow:/mnt/bhyve-script # ls /dev/vmm/
> > >>
> > >> lin0
> > >>
> > >> root at bulow:/mnt/bhyve-script # bhyvectl --destroy --vm=lin0
> > >>
> > >> errno = 22
> > >>
> > >>
> > >> On Thu, Jan 23, 2014 at 3:44 PM, Matthias Gamsjager <
> > mgamsjager at gmail.com>wrote:
> > >>
> > >>> Could you elaborate what you did to get it working?
> > >>>
> > >>>
> > >>> On Thu, Jan 23, 2014 at 3:41 PM, Jonas Bülow <jonas.bulow at gmail.com
> > >wrote:
> > >>>
> > >>>> Success. I have a working ubuntu 13.10 running under bhyve on
> FreeBSD
> > >>>> 10.0-RELEASE.
> > >>>>
> > >>>>
> > >>>>
> > >>>> /J
> > >>>>
> > >>>>
> > >>>> On Thu, Jan 23, 2014 at 11:37 AM, Aryeh Friedman
> > >>>> <aryeh.friedman at gmail.com>wrote:
> > >>>>
> > >>>> >
> > >>>> >
> > >>>> >
> > >>>> > On Thu, Jan 23, 2014 at 5:05 AM, Jonas Bülow <
> jonas.bulow at gmail.com
> > >>>> >wrote:
> > >>>> >
> > >>>> >> I do use the precanned bhyve-script "vm0" (if that is what you
> > mean).
> > >>>> It
> > >>>> >> explicitly mentions linux support. bhyveload can't be user for
> > other
> > >>>> images
> > >>>> >> than FreeBSD so I don't know how to use your script for a linux
> OS
> > >>>> >>
> > >>>> >
> > >>>> > Since I have not used bhyve-grub personally yet I can not
> recommend
> > >>>> what
> > >>>> > the correct loader command line but if bhyve-script is in anyway
> > >>>> related to
> > >>>> > vmrun.sh I would be highly suspicious of any command lines you
> > manage
> > >>>> to
> > >>>> > extract from it since they are often not the ones actual called.
> > >>>> >
> > >>>> >
> > >>>> >>
> > >>>> >> I assume the image is file backed as I have not assigned a
> > dedicated
> > >>>> >> partition to the vm (and I don't know how to do that).
> > >>>> >>
> > >>>> >
> > >>>> > You can keep them anywhere no need for a special volume (mater of
> > fact
> > >>>> I
> > >>>> > was just reusing the name of one of the diretoriess that PC
> creates
> > >>>> during
> > >>>> > install even without it being a different vol [just assumes any FS
> > >>>> > boundaries are the host OS's problem)... you can put the disk
> image
> > any
> > >>>> > where (your home directory would be fine for example)
> > >>>> >
> > >>>> _______________________________________________
> > >>>> freebsd-virtualization at freebsd.org mailing list
> > >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> > >>>> To unsubscribe, send any mail to "
> > >>>> freebsd-virtualization-unsubscribe at freebsd.org"
> > >>>>
> > >>>
> > >>>
> > >>
> > > _______________________________________________
> > > freebsd-virtualization at freebsd.org mailing list
> > > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> > > To unsubscribe, send any mail to "
> > freebsd-virtualization-unsubscribe at freebsd.org"
> >
> _______________________________________________
> freebsd-virtualization at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe at freebsd.org"
>



-- 
Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org

More information about the freebsd-virtualization mailing list