svn commit: r355724 - in head: sys/amd64/include sys/amd64/vmm/amd sys/amd64/vmm/intel usr.sbin/bhyve

[John Baldwin]

On 12/13/19 11:21 AM, John Baldwin wrote:
> Author: jhb
> Date: Fri Dec 13 19:21:58 2019
> New Revision: 355724
> URL: https://svnweb.freebsd.org/changeset/base/355724
> 
> Log:
>   Support software breakpoints in the debug server on Intel CPUs.
>   
>   - Allow the userland hypervisor to intercept breakpoint exceptions
>     (BP#) in the guest.  A new capability (VM_CAP_BPT_EXIT) is used to
>     enable this feature.  These exceptions are reported to userland via
>     a new VM_EXITCODE_BPT that includes the length of the original
>     breakpoint instruction.  If userland wishes to pass the exception
>     through to the guest, it must be explicitly re-injected via
>     vm_inject_exception().
>   
>   - Export VMCS_ENTRY_INST_LENGTH as a VM_REG_GUEST_ENTRY_INST_LENGTH
>     pseudo-register.  Injecting a BP# on Intel requires setting this to
>     the length of the breakpoint instruction.  AMD SVM currently ignores
>     writes to this register (but reports success) and fails to read it.
>   
>   - Rework the per-vCPU state tracked by the debug server.  Rather than
>     a single 'stepping_vcpu' global, add a structure for each vCPU that
>     tracks state about that vCPU ('stepping', 'stepped', and
>     'hit_swbreak').  A global 'stopped_vcpu' tracks which vCPU is
>     currently reporting an event.  Event handlers for MTRAP and
>     breakpoint exits loop until the associated event is reported to the
>     debugger.
>   
>     Breakpoint events are discarded if the breakpoint is not present
>     when a vCPU resumes in the breakpoint handler to retry submitting
>     the breakpoint event.
>   
>   - Maintain a linked-list of active breakpoints in response to the GDB
>     'Z0' and 'z0' packets.
>   
>   Reviewed by:	markj (earlier version)
>   MFC after:	2 months
>   Differential Revision:	https://reviews.freebsd.org/D20309

As the manpage notes, there is a pretty large caveat with using breakpoints.
The debugger wants to single-step over a breakpoint after replacing the
original instruction before resuming.  However, the latency between a
breakpoint firing and the user responding in the debugger is such that a
timer interrupt has triggered by the time the vCPU resumes.  Thus, the
single step stops in the first instruction of the interrupt handler.  The
debugger then does the user's requested continue which finishes the interrupt
handler and trips the breakpoint again at the original location when the
interrupt handler returns.  The effect is that doing a continue after a
breakpoint never makes forward progress.  One workaround is to disable
the current breakpoint and use 'until' to set a temporary breakpoint at
the next line in the source.  You can then re-enable the original breakpoint
and continue.

I've thought about various ways to fix this, but they all have downsides.
One way is to clear IF in %eflags while stepping, but then you have to
emulate pushf and possibly popf.  Another option might be to add new commands
to pause and unpause timer devices and pause timers when the vCPUs all exit
and re-enable when either doing a continue or for the duration of a step.
The latter approach feels a bit more of what you want, but it has other
potential downsides, like time jumps in the guest, etc.

-- 
John Baldwin