Detect of BHyve VM was powered off or rebooted?

Neel Natu neelnatu at gmail.com
Mon Feb 10 00:14:16 UTC 2014


Hi Aryeh,

On Sun, Feb 9, 2014 at 2:34 PM, Aryeh Friedman <aryeh.friedman at gmail.com> wrote:
> On Sun, Feb 9, 2014 at 5:18 PM, Craig Rodrigues <rodrigc at freebsd.org> wrote:
>
>>
>>
>>
>> On Sun, Feb 9, 2014 at 1:32 PM, Aryeh Friedman <aryeh.friedman at gmail.com>wrote:
>>
>>>
>>>
>>>
>>> On Sun, Feb 9, 2014 at 4:11 PM, Craig Rodrigues <rodrigc at freebsd.org>wrote:
>>>
>>>> Hi,
>>>>
>>>> I posted some rc.d scripts that I am using to boot a BHyve VM
>>>> and send the output to a serial console using the /dev/nmdm
>>>> driver:
>>>>
>>>>
>>>> http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/002040.html
>>>>
>>>> It works quite well.  There is some things I would like to improve,
>>>> and would like some advice on the best way to do it.
>>>>
>>>> (1)  If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
>>>>       then I do not want to automatically restart the VM in the script.
>>>>       User should manually:  service bhyvevm start
>>>>
>>>> (2)  If the VM was powered down, via shutdown -p, or halt -p,
>>>>        then in my script I do not want to restart the VM in the script.
>>>>        User should manually:  service bhyvevm start
>>>>
>>>> (3)   If the VM was rebooted via "reboot" or "shutdown -r",
>>>>        then I *do* want the script to restart the VM.
>>>>
>>>> I think if I change my start_vm.sh script to do something like:
>>>>
>>>>
>>>>
>>>> (
>>>>  while [ -e /dev/vmm/${VM} ]; do
>>>>     /usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s
>>>> 1:0,lpc
>>>> -s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A}
>>>> ${VM}"
>>>>  done
>>>>
>>>> )  &
>>>>
>>>>
>>>> then this might cover cases (1) and (3), but what will cover
>>>> case (2)?
>>>>
>>>> Thanks for any advice.
>>>>
>>>
>>> The options you gave (which are really the only ones available) do not
>>> distinguish between the reason for the termination of the instance of bhyve
>>> pointing to /dev/vmm/XXX  (it just does a normal termination).
>>>
>>
>> Did you play with any of these flags to bhyvectl?
>>
>>        [--get-vmcs-exit-ctls]
>>        [--get-vmcs-exit-reason]
>>        [--get-vmcs-exit-qualification]
>>        [--get-vmcs-exit-interruption-info]
>>
>> If I do:
>> bhyvectl --get-vmcs-exit-reason --vm vm1
>>
>> I get:
>> vmcs_exit_reason[0]     0x000000000000001e
>>
>> If I look at:
>>
>>
>> http://svnweb.freebsd.org/base/head/sys/amd64/vmm/intel/vmcs.h?view=markup#l310
>>
>> we have:
>> #define EXIT_REASON_INOUT 30
>>
>
> You will likely need to detect the following ones also (all for different
> reasons that may or may not have anything to do with an issue with the VM
> instead of some host [OS or hardware] issue):
>
> #define EXIT_REASON_EXT_INTR 1
> #define EXIT_REASON_INIT 3
> #define EXIT_REASON_HLT 12
> #define EXIT_REASON_CPUID 10
> #define EXIT_REASON_VMLAUNCH 20
>

These are VMCS exit reasons that have nothing to do whatsoever with
the guest wanting to reboot or power itself off.

best
Neel

>
>> Linux has this stuff also:
>> http://lxr.free-electrons.com/source/arch/x86/include/asm/vmx.h?v=3.1#L250
>>
>> so I guess these values are defined in some Intel manual.
>>
>>
>>
>>> In my playing with this for almost a month the only solution seems to be
>>> put a trigger on instance shutdown via whatever method to leave a "cookie"
>>> file behind on the host (note I have not nor plan to do this in any of my
>>> own work).
>>>
>>
>> Yes, I can see why this might need to be done.  I might need to do that
>> myself.  However, if bhyvectl can be used to provide adequate status
>> information as to why the VM exited, I might be able to avoid this.
>>
>> It would be nicer though if /usr/sbin/bhyve returned
>> the EXIT_REASON in its status code.
>>
>>
>>
>>>
>>>   An other question is it wise to auto-reboot regardless of reason of the
>>> hypervisor termination?
>>>
>>
>> I think the answer is "it depends".  If someone
>> does "reboot" or "shutdown -r", I think it is reasonable to restart the
>> VM.  If someone does "halt -p" or "shutdown -p", I think it is reasonable
>> to not restart the VM.  For any other error condition, I think it is
>> reasonable to not restart the VM.
>>
>> I think that there is no  single answer for all users and all applications.
>> In my case, I am running BHyve VM's as "headless" which
>> I access via the /dev/nmdm driver.  If the VM reboots normally, I want it
>> to restart.
>>
>
> It seems almost any boot failure would be host side for example if there is
> no SSH access (which if the instance is never logged into then it is likely
> a host issue) then it is highly unlikely the VM is in bootable form (this
> is after a manual reboot of it)... an other reason (just learned the
> hardway) is the user uses a mismatch between backing file formats (my case
> using raw images with ahci-hd)
> --
> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org
> _______________________________________________
> freebsd-virtualization at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe at freebsd.org"


More information about the freebsd-virtualization mailing list