bhyve: svm (amd-v) update

Fri May 16 21:21:56 UTC 2014

Hi Anish,

Anish wrote:
> If patches looks good to you, we can submit it. I have been testing it on
> Phenom box which lacks some of newer SVM features.

Your patch applied cleanly to the working copy of the "bhyve_svm"-project. I was then able to merge with HEAD 
(using "theirs-full" on one file) and compile the kernel. So, to me it looks OK to commit.

Unfortunately, I am still not able to boot CentOS 6.5 using my Phenom 1055T. It produces 200% load on the
host CPU, and the emulated machine generates endlessly:
=======================================================================================
BUG: soft lockup - CPU#0 stuck for 67s! [swapper:1]
Modules linked in:
CPU 0
Modules linked in:

Pid: 1, comm: swapper Not tainted 2.6.32-431.el6.x86_64 #1   BHYVE
RIP: 0010:[<ffffffff81c5496d>]  [<ffffffff81c5496d>] rc_is_bit_0+0x3a/0x69
RSP: 0018:ffff88013e79dca0  EFLAGS: 00000a96
RAX: 000000000000009c RBX: ffff88013e79dcc0 RCX: ffff880004bdcc7c
RDX: 00000000002f9dee RSI: ffffc9000402c538 RDI: ffff88013e79ddb0
RBP: ffffffff8100b9ce R08: ffffc9000402c788 R09: ffffffff81de32b8
R10: 0000000000000003 R11: 0000000000000000 R12: 0000000000000003
R13: ffffffff81157602 R14: ffff88013e79dc20 R15: 00000000000000d2
FS:  0000000000000000(0000) GS:ffff880028200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 0000000001a85000 CR4: 00000000000007b0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper (pid: 1, threadinfo ffff88013e79c000, task ffff88013e79b500)
Stack:
 ffffc9000402c644 00000000000000e5 00000000000003b1 ffff88013e79ddb0
<d> ffff88013e79dcf0 ffffffff81c549b7 ffffffffffffffb6 0000000000000000
<d> 0000000000000000 ffffc9000402c000 ffff88013e79de30 ffffffff81c554e2
Call Trace:
 [<ffffffff81c549b7>] ? rc_get_bit+0x1b/0x79
 [<ffffffff81c554e2>] ? unlzma+0xa42/0xc67
 [<ffffffff81c28ab9>] ? flush_buffer+0x0/0xa3
 [<ffffffff811bb9cb>] ? do_utimes+0xdb/0x170
 [<ffffffff812827a0>] ? nofill+0x0/0x10
 [<ffffffff81c29776>] ? unpack_to_rootfs+0x167/0x27a
 [<ffffffff81c28929>] ? error+0x0/0x17
 [<ffffffff812a6725>] ? pci_get_subsys+0x35/0x40
 [<ffffffff81c2992b>] ? populate_rootfs+0x0/0xd3
 [<ffffffff81c29986>] ? populate_rootfs+0x5b/0xd3
 [<ffffffff8100204c>] ? do_one_initcall+0x3c/0x1d0
 [<ffffffff81c268e4>] ? kernel_init+0x29b/0x2f7
 [<ffffffff8100c20a>] ? child_rip+0xa/0x20
 [<ffffffff81c26649>] ? kernel_init+0x0/0x2f7
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
Code: ff ff ff 00 77 35 48 8b 47 18 48 39 47 08 72 0d 48 89 75 e8 e8 95 ff ff ff 48 8b 75 e8 48 8b 4b 08 c1 63 28 08 8b 53 24 0f b6 01 <48> 83 c1 01 c1 e2 08 48 89 4b 08 09 d0 89 43 24 0f b7 06 8b 53
Call Trace:
 [<ffffffff81c549b7>] ? rc_get_bit+0x1b/0x79
 [<ffffffff81c554e2>] ? unlzma+0xa42/0xc67
 [<ffffffff81c28ab9>] ? flush_buffer+0x0/0xa3
 [<ffffffff811bb9cb>] ? do_utimes+0xdb/0x170
 [<ffffffff812827a0>] ? nofill+0x0/0x10
 [<ffffffff81c29776>] ? unpack_to_rootfs+0x167/0x27a
 [<ffffffff81c28929>] ? error+0x0/0x17
 [<ffffffff812a6725>] ? pci_get_subsys+0x35/0x40
 [<ffffffff81c2992b>] ? populate_rootfs+0x0/0xd3
 [<ffffffff81c29986>] ? populate_rootfs+0x5b/0xd3
 [<ffffffff8100204c>] ? do_one_initcall+0x3c/0x1d0
 [<ffffffff81c268e4>] ? kernel_init+0x29b/0x2f7
 [<ffffffff8100c20a>] ? child_rip+0xa/0x20
 [<ffffffff81c26649>] ? kernel_init+0x0/0x2f7
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
=======================================================================================

Additionally, It produces a lot of MSR requests:
=======================================================================================
May 16 09:32:03 10.255.255.96 kernel: emulate_rdmsr 0xc0010015
May 16 09:32:18 10.255.255.96 kernel: emulate_rdmsr 0x1b
May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010112
May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010048
May 16 09:32:23 10.255.255.96 kernel: emulate_wrmsr 0xc0010048
May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0x8b
May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010140
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010000
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010001
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010002
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010003
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010004
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010004
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x1b
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010048
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010048
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x8b
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:25 10.255.255.96 last message repeated 314 times
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x1b
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010048
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010048
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x8b
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:25 10.255.255.96 last message repeated 49 times
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004
May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:25 10.255.255.96 last message repeated 885 times
May 16 09:32:25 10.255.255.96 kernel: 010055
May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:25 10.255.255.96 last message repeated 4820 times
May 16 09:32:26 10.255.255.96 kernel: 010055
May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:26 10.255.255.96 last message repeated 4364 times
May 16 09:32:26 10.255.255.96 kernel: 010055
May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:26 10.255.255.96 last message repeated 25 times
May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f
May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f
May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f
May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f
May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:26 10.255.255.96 last message repeated 391 times
May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f
May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f
May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:34 10.255.255.96 last message repeated 73074 times
May 16 09:32:34 10.255.255.96 kernel: 010055
May 16 09:32:34 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:35 10.255.255.96 last message repeated 14648 times
May 16 09:32:35 10.255.255.96 kernel: 010055
May 16 09:32:35 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:35 10.255.255.96 last message repeated 8098 times
May 16 09:32:36 10.255.255.96 kernel: 010055
May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:36 10.255.255.96 last message repeated 7895 times
May 16 09:32:36 10.255.255.96 kernel: 010055
May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:36 10.255.255.96 last message repeated 8272 times
May 16 09:32:36 10.255.255.96 kernel: 010055
May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:37 10.255.255.96 last message repeated 8696 times
May 16 09:32:37 10.255.255.96 kernel: 010055
May 16 09:32:37 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:37 10.255.255.96 last message repeated 12333 times
May 16 09:32:38 10.255.255.96 kernel: 010055
May 16 09:32:38 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:41 10.255.255.96 last message repeated 30370 times
May 16 09:32:41 10.255.255.96 kernel: emulate_rdmsr 0xc001103a
May 16 09:32:41 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:49 10.255.255.96 last message repeated 85577 times
May 16 09:32:49 10.255.255.96 kernel: 010055
May 16 09:32:49 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:56 10.255.255.96 last message repeated 9534 times
May 16 09:32:57 10.255.255.96 kernel: 010055
May 16 09:32:57 10.255.255.96 kernel: emulate_rdmsr 0xc0010055
May 16 09:32:58 10.255.255.96 last message repeated 17524 times
[and so on]
=======================================================================================

I'd love to see CentOS perfectly running on my Phenom as it runs perfectly on an Intel i3.

If you need any further information/debug, please let me know...

TIA and regards,
Nils