Re: Can we "pause" in loader and ddb>?

On Mon, Nov 21, 2022 at 02:06:47PM -0500, Mark Johnston wrote:
> On Mon, Nov 21, 2022 at 06:42:44PM +0000, Bjoern A. Zeeb wrote:
> > Hi,
> > 
> > having a VM sitting (a) in loader prompt or (b) in ddb> makes the fans
> > go loud on a test-laptop which makes one wonder if we can avoid that.
> > Note -- I assume also on real HW though that's less likely observered
> > here.
> > 
> > For example I am sitting in a 4 vCPU bhyve current in ddb> and two
> > threads are at 100% on the base system.  Do we need to heat up the
> > planet doing that or are there alternatives?  I haven't looke dta the
> > code in ages and I assume we need to poll in these situations on the
> > console and for interactivity often enough?  Can we "pause"?   And why
> > would two [v]CPUs be at 100% and not just one as I would expect all but
> > one to be stopped?
> 
> When DDB is active, one CPU is going to be spinning in cncheckc()
> waiting for input, and the others will be in cpustop_handler().  For the
> latter, there is an option to use MWAIT/MONITOR to pause the CPU instead
> of calling pause in a loop, but bhyve hides the MWAIT/MONITOR CPU
> capability from guests (I'm not sure exactly why), so they just execute
> "pause" in a loop, which won't help the apparent CPU usage.
We do not want the guest thread eating CPU time in MWAIT, it is the host
scheduler that should decide that there is nothing to do and schedule
idle thread.

In other words, MWAIT should be emulated, perhaps by revoking the monitoring
page on MONITOR, and waking up the MWAIT-ing thread on write to the armed
range.

Additional complication is that the range to arm is usually equal to the cache
line, doing dumb emulation by interpreting any write to the monitor page
might be too harsh.

There is also non-trivial cooperation with atomic interrupt delivery.

These complexities are perhaps the reason to not enable M* for guests.

> 
> Note that when the kernel panics, cpustop_handler() will execute "hlt",
> which should result in a vmexit if you started bhyve with -H, and you'll
> see only one vCPU thread consuming 100% when sitting at the DDB prompt.
> But when you enter DDB without a panic, we need some mechanism to resume
> the stopped CPUs when exiting DDB.