Re: Can we "pause" in loader and ddb>?

On Mon, 21 Nov 2022, Mark Johnston wrote:

Hi,

> On Mon, Nov 21, 2022 at 09:37:36PM +0200, Konstantin Belousov wrote:
>> On Mon, Nov 21, 2022 at 02:06:47PM -0500, Mark Johnston wrote:
>>> On Mon, Nov 21, 2022 at 06:42:44PM +0000, Bjoern A. Zeeb wrote:
>>>> Hi,
>>>>
>>>> having a VM sitting (a) in loader prompt or (b) in ddb> makes the fans
>>>> go loud on a test-laptop which makes one wonder if we can avoid that.
>>>> Note -- I assume also on real HW though that's less likely observered
>>>> here.
>>>>
>>>> For example I am sitting in a 4 vCPU bhyve current in ddb> and two
>>>> threads are at 100% on the base system.  Do we need to heat up the
>>>> planet doing that or are there alternatives?
>
> I forgot to mention: in this situation my "solution" is to attach gdb to
> the running VM, which I usually want to do anyway.  This causes all vCPU
> threads to pause.
>
>>>> I haven't looke dta the
>>>> code in ages and I assume we need to poll in these situations on the
>>>> console and for interactivity often enough?  Can we "pause"?   And why
>>>> would two [v]CPUs be at 100% and not just one as I would expect all but
>>>> one to be stopped?
>>>
>>> When DDB is active, one CPU is going to be spinning in cncheckc()
>>> waiting for input, and the others will be in cpustop_handler().  For the
>>> latter, there is an option to use MWAIT/MONITOR to pause the CPU instead
>>> of calling pause in a loop, but bhyve hides the MWAIT/MONITOR CPU
>>> capability from guests (I'm not sure exactly why), so they just execute
>>> "pause" in a loop, which won't help the apparent CPU usage.
>> We do not want the guest thread eating CPU time in MWAIT, it is the host
>> scheduler that should decide that there is nothing to do and schedule
>> idle thread.
>>
>> In other words, MWAIT should be emulated, perhaps by revoking the monitoring
>> page on MONITOR, and waking up the MWAIT-ing thread on write to the armed
>> range.
>>
>> Additional complication is that the range to arm is usually equal to the cache
>> line, doing dumb emulation by interpreting any write to the monitor page
>> might be too harsh.
>>
>> There is also non-trivial cooperation with atomic interrupt delivery.
>>
>> These complexities are perhaps the reason to not enable M* for guests.
>
> I see.  Maybe instead we could allow stopped CPUs to execute "hlt" with
> interrupts disabled, and when exiting DDB, the last CPU resumes all of
> the others by raising NMIs.  Then only one vCPU thread will be consuming
> host CPU time when in DDB.
>
>>> Note that when the kernel panics, cpustop_handler() will execute "hlt",
>>> which should result in a vmexit if you started bhyve with -H, and you'll
>>> see only one vCPU thread consuming 100% when sitting at the DDB prompt.
>>> But when you enter DDB without a panic, we need some mechanism to resume
>>> the stopped CPUs when exiting DDB.

Nonwithstanding, the problem also is loader and the problem probably
also is on a bare metal so focusing on bhyve is certainly a partial win
but only part of the solution whihc ideally works in both situations.

-- 
Bjoern A. Zeeb                                                     r15:7