Debugging high(ish) CPU usage on an idle VM

[Dustin Marquess]

I'm running an 11.0-CURRENT (from exactly a week ago) on a 32-core
(16-module) AMD Bulldozer system.

Guest is NetBSD 7.99.21 HEAD, with timecounter hard-forced to
ACPI-safe instead of hpet (HPET seemed unstable, although HPET is
being used in the host FreeBSD).

I start bhyve using a simple script:

#!/bin/sh
RAM=8192M
VM=netbsd
VMROOT=/usr/local/vm
CPUS=8

grub-bhyve -m ${VMROOT}/${VM}/device.map -d ${VMROOT}/${VM} -M ${RAM} ${VM}
exec bhyve -A -H -P -w -u -s 0:0,amd_hostbridge -s 1:0,lpc -s
2:0,virtio-net,tap0 -s 3:0,virtio-blk,/dev/zvol/pool/vm/${VM},nocache
-s 4:0,virtio-rnd -l com1,stdio -c ${CPUS} -m ${RAM} ${VM}

While the guest is entirely idle:

37975 root          1  20    0 13084K  3016K wait   31   0:00   0.00% sh
load averages:  0.00,  0.00,  0.00;               up 1+22:00:40

03:34:40
18 processes: 16 sleeping, 2 on CPU
CPU states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Memory: 923M Act, 6996K Wired, 12M Exec, 900M File, 5911M Free
Swap: 8195M Total, 8195M Free

Bhyve itself seems to go bounce around between ~6% CPU usage to 18%:

last pid: 37995;  load averages:  0.22,  0.38,  0.41
                                                 up 5+20:05:17
21:26:35
24 processes:  1 running, 23 sleeping
CPU:  0.0% user,  0.0% nice,  0.5% system,  0.1% interrupt, 99.4% idle
Mem: 2572M Active, 5735M Inact, 19G Wired, 61M Buf, 35G Free
ARC: 14G Total, 3656M MFU, 10G MRU, 16K Anon, 128M Header, 566M Other
Swap: 64G Total, 64G Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
18577 root         18  20    0  9247M  5048M kqread 15 427:55  18.65% bhyve

Any ideas to help troubleshoot where the CPU usage is going, DTrace?

Thanks!
-Dustin