CFT: Reload LDTR after #VMEXIT on AMD-v in bhyve

Mike Tancsa mike at
Thu Oct 11 20:40:19 UTC 2018

On 10/11/2018 3:02 PM, John Baldwin wrote:
> Can someone using bhyve on an AMD host test this patch?  Just booting a
> guest to multiuser is probably sufficient testing:
> Thanks.

well, if I let the box boot fully and then load the vmm.ko, all is good.
But if I load it from /boot/loader.conf, I get a panic at boot up (img

But other than that, it works fine.

0{ryzenbsd12}# fetch -o p
size of remote file is not known
p                                                     1618  B   15
MBps    00s
0{ryzenbsd12}# patch < p
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
|From 97323364e196900548f5293ac97bfb22b8a2ba72 Mon Sep 17 00:00:00 2001
|From: John Baldwin <jhb at>
|Date: Tue, 9 Oct 2018 14:49:37 -0700
|Subject: [PATCH] Reload the LDT selector after an AMD-v #VMEXIT.
|cpu_switch() always reloads the LDT, so this can only affect the
|hypervisor process itself.  Fix this by explicitly reloading the host
|LDT selector after each #VMEXIT.  The stock bhyve process on FreeBSD
|never uses a custom LDT, so this change is cosmetic.
| sys/amd64/vmm/amd/svm.c | 13 +++++++++++++
| 1 file changed, 13 insertions(+)
|diff --git a/sys/amd64/vmm/amd/svm.c b/sys/amd64/vmm/amd/svm.c
|index 2597bf9775706..c420db550bc7e 100644
|--- a/sys/amd64/vmm/amd/svm.c
|+++ b/sys/amd64/vmm/amd/svm.c
Patching file sys/amd64/vmm/amd/svm.c using Plan A...
Hunk #1 succeeded at 1940.
Hunk #2 succeeded at 2025.
Hunk #3 succeeded at 2064.

I confirmed prior to the patch I could run stock 11.2R  i386 and amd64
guest images on 12.0-ALPHA9 FreeBSD 12.0-ALPHA9 r339287 as the hypervisor

CPU: AMD Ryzen 5 1600X Six-Core Processor            (3593.34-MHz
K8-class CPU)
  Origin="AuthenticAMD"  Id=0x800f11  Family=0x17  Model=0x1  Stepping=1
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  Structured Extended
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr>
  SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics

with the patched version

ivhd0: <AMD-Vi/IOMMU ivhd with EFR> on acpi0
ivhd0: Flag:b0<IotlbSup,Coherent>
ivhd0: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0
ivhd0: Extended features[31:0]:22294ada<PPRSup,NXSup,GTSup,IASup> HATS =
0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1
DualPortLogSup = 0x2 DualEventLogSup = 0x2
ivhd0: Extended features[62:32]:f77ef<USSup> Max PASID: 0x2f
DevTblSegSup = 0x3 MarcSup = 0x1
ivhd0: supported paging level:7, will use only: 4
ivhd0: device range: 0x0 - 0xffff
ivhd0: PCI cap 0x190b640f at 0x40 feature:19<IOTLB,EFR,CapExt>

Tested with

./ -c 4 -m 4096M -t tap0 -d FreeBSD-11.2-RELEASE-amd64.raw amd64
./ -c 4 -m 4096M -t tap1 -d FreeBSD-11.2-RELEASE-i386.raw i386

I sent a jpg of the crash in a separate image as it was rejected from the mailing list.


Mike Tancsa, tel +1 519 651 3400 x203
Sentex Communications, mike at
Providing Internet services since 1994
Cambridge, Ontario Canada   

More information about the freebsd-virtualization mailing list