VMX exit reason=33 and general userboot.so questions

[Fabian Freyer]

Hi Peter,

thanks for your response!

On 21 Feb 2018, at 17:59, Peter Grehan wrote:
>  >        exit_reason     33
>
>  From the Intel SDM, vol 3B Appendix C, this error is "VM-entry failure due to invalid guest state".

Yes, I’m currently going through SDM, vol 3B, section 26.3, and

>  These errors can be difficult to debug given the large amount of guest state involved :(

definitely accurately describes the experience. That said, I’m going through each check and trying to see what is going wrong.

>  However, looking at the state from your dump:
>
>  > tr desc[0]      0x0000000000000000/0x00000000/0x00000000
>
>  I believe you will have to set this. Here's the comment and relevant code fragment from grub2-bhyve grub-core/kern/emu/bhyve_hostif.c:grub_emu_bhyve_boot32()
>
>   /*
>    * XXX TR is pointing to null selector even though we set the
>    * TSS segment to be usable with a base address and limit of 0.
>    * Has to be 8b or vmenter will fail
>    */
>   desc_access = 0x0000008b;
>   assert(vm_set_desc(bhyve_ctx, 0, VM_REG_GUEST_TR, 0x1000, 0x67,
>                      desc_access) == 0);
>
>  grub2-bhyve has been able to load/boot multiboot images, so I suspect the register settings in grub_emu_bhyve_boot32() are a good place to start from.

Thanks. I’ve tried setting all registers (especially) the segment registers as grub2-bhyve does, but I still don’t seem to be able to boot correctly. Fixing the TR descriptor didn’t do it.

Here’s a list of the descriptors I set up:

ds desc[0]      0x0000000000000000/0xffffffff/0x00000093
es desc[0]      0x0000000000000000/0xffffffff/0x00000093
fs desc[0]      0x0000000000000000/0xffffffff/0x00000093
gs desc[0]      0x0000000000000000/0xffffffff/0x00000093
ss desc[0]      0x0000000000000000/0xffffffff/0x00000093
cs desc[0]      0x0000000000000000/0xffffffff/0x0000009b
tr desc[0]      0x0000000000001000/0x00000067/0x0000008b
ldtr desc[0]    0x0000000000000000/0x0000ffff/0x00010082


gdtr points to a default gdt as set up in grub2-bhyve:

static uint16_t bhyve_gdt[] = {
  0x0000, 0x0000, 0x0000, 0x0000,       /* Null */
  0x0000, 0x0000, 0x0000, 0x0000,       /* Null #2 */
  0xffff, 0x0000, 0x9a00, 0x00cf,       /* code */
  0xffff, 0x0000, 0x9200, 0x00cf,       /* data */
  0x0000, 0x0000, 0x8900, 0x0080        /* tss */
};

gdtr[0]         0x0000000000100161/0x00000027
idtr[0]         0x0000000000000000/0x00000000

The rest is set up as by grub2-bhyve:
cs[0]           0x0008
ds[0]           0x0010
es[0]           0x0010
fs[0]           0x0010
gs[0]           0x0010
ss[0]           0x0010
tr[0]           0x0000
ldtr[0]         0x0000

I’m assuming entry_ctls[0] = 0x91fb is the VM-entry control, but is it the value described by Table 24-7 of the SDM Volume 3B? Is this constant for the first entry after running bhyveload?  That is, when writing some compliance checks against section 26.3.1, can I just hardcode that value?

I’m starting to dig through the bhyverun code, but I’m still pretty new to VMX stuff, so I am ending up with more questions than answers.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 882 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20180221/aaec723f/attachment.sig>