dtrace not working on bhyve VM without invariant_tsc

[Eric van Gyzen]

> On Dec 9, 2019, at 8:27 PM, Ryan Stone <rysto32 at gmail.com> wrote:
> 
> I have a bhyve VM guest on my laptop where dtrace just constantly
> aborts whenever I try to use it:
> 
> [rstone at ebpf dtrace]sudo dtrace -s fdcopy.d
> Assertion failed: (buf->dtbd_timestamp >= first_timestamp), file
> /usr/home/rstone/git/bsd-worktree/ebpf-import/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c,
> line 3026.
> Abort trap
> 
> I believe that the problem is caused by dtrace unconditionally using
> rdtsc() to implement dtrace_gethrtime(), assuming that the values will
> be stable for a given CPU.  The VM's vcpus seem to be getting migrated
> frequently.
> 
> Should dtrace instead be using the system timecounter?  That should
> stand a much better chance of being monotonically increasing.

I’ve seen TSC issues with OneFS under bhyve on certain CPUs.  Pinning the VM to CPUs 1-N (i.e. avoiding CPU 0) worked around it.  You might try that as a workaround.

Eric