bhyve: Detecting that a guest kernel has booted

[Rodney W. Grimes]

> On 2019-03-11T13:27:23 -0400
> Shawn Webb <shawn.webb at hardenedbsd.org> wrote:
> >
> > Both FreeBSD and Linux supports virtio_console(4). I have no idea
> > about OpenBSD, but I'm sure they'd be open to an implementation if
> > asked.
> 
> Right.
> 
> > 
> > The NFS solution would work, but it would be somewhat fragile. What
> > happens when a VM crashes? What happens when the host crashes?
> 
> At least in my case:
> 
> If the VM crashes, it'll be restarted by a process supervisor (runit,
> here).

I think ping would suffice to make that determination?

> If the host crashes, I likely have bigger problems. In any case, I
> think that's still fine because all the host would care about is if the
> guest's file was touched more recently than the last time the host
> tried to start a bhyve process for it.

You could inside the vm simply touch /tmp/foo from cron and from the host
see that the diskimage last modified time updated.  Assuming some cacheing
does not get in the way.

> I suppose I should elaborate a bit: I do have monitoring via Prometheus
> in place, but I'd like to try to stagger VM startups a little as
> starting up a lot of them in parallel on boot tends to overwhelm the
> machine slightly. Once they're all up and running in a steady state,
> things are fine. I would typically stagger the startup of
> ordinary services (inside a jail, for example) by using dependencies in
> runit - it has a facility to pause a service until a dependent service
> has been started. Unfortunately, that can't work in this case because
> once the bhyve process has been started, the host can't tell if/when
> the guest has actually fully started up. As far as runit is concerned,
> the service is up and so any dependent services should be started too.

The package vmbhyve has starggered startup in a specific ordered list
implemented.  It is all writtin in /bin/sh, so easy to adapt.

> Mark Raynsford | http://www.io7m.com
-- 
Rod Grimes                                                 rgrimes at freebsd.org