suspend/resume on BHyVe

[Iori YONEJI]

Hello.

I'm thinking of adding suspend/resume features on BHyVe.

For this, I think those below must be implemented.

  - virtual machine state command interface
  - saving registers per CPU
  - dumping physical memory
  - saving virt-io and other device emulation state


To save registers, the sysctl used in bhyvectl (vmmctl command
previously) is helpful,
however, it's interface is no good because getting register value
cause a sysctl call
so to context-switch per one register, and for getting all registers,
it's not efficient.
I think it's more preferable to make a struct to set or unset boolean
fields per register,
to tell which registers kernel should return, and kernel returns those
state with struct vmxctx.

struct vmxctx is such.
 struct vmxctx {
        register_t      tmpstk[32];             /* vmx_return() stack */
        register_t      tmpstktop;

        register_t      guest_rdi;              /* Guest state */
        register_t      guest_rsi;
                :
                :


And, considering memory dump, /dev/vmm/vmname is a file that is a map
of guest memory,
so memory dump doesn't seem hard, just stop vm, write back all guest
cache, and copy
memory file to a regular file.

Finally, I don't know much about device state, but I think there must
some state to be saved, like
network stack.

I'm not sure I wrote former, so I appreciate your ideas and suggestions.

Thanks, Iori.