[RFC] Adding a Rados block driver to bhyve

[Alan Somers]

On Mon, Mar 9, 2020 at 4:32 AM Willem Jan Withagen <wjw at digiware.nl> wrote:

> Hi all,
>
> And sorry for crosspoing three groups, but the answer can/could be a mix
> of things to do in these three areas.
>
> I have a prototype of bhyve running on Rados/Ceph working:
>      https://github.com/freebsd/freebsd/pull/426
>
> But there are a few catches on how to get it in the FreeBSd sources...
>
> 1) Easiest would be to just compile it in with the code of the current
> bhyve.
>      That will require librados/librbd libraries...
>      Ceph of this purpose is LGPL2/3 and could go into contrib.
>      In this case bhyve will hold the rbd-driver by default and a user
> does not
>      need to do anything by himself
>      But I have the feeling that this is the most unwanted scenario
>
> 2) User first installs a Ceph package and FreeBSD sources, and then
> recompiles
>      bhyve with the option BHYVE_RBD.
>      And then reinstalls this new version as bhyve or bhyve-rbd in
> /usr/sbin
>
> 3) Create a bhyve-rbd port.
>      Problem with that is that it will require the FreeBSD source tree
> for the
>      bhyve sources, but there is no Ports option for that?
>      Or bhyve sources are manually copied into the port. And then
>      try to keep these sources up to date.
>      Then compile and install a bhyve-rbd into /usr/local/sbin
>
> 4) Create a bhyve-blockrbd port.
>      This is much like 3) but instead of building a bhyve-rbd executable,
>      it delivers a libblockrbd.so that is dynamically loadable by the
>      standaard bhyve that comes with base.
>
>      For this bhyve needs to be extended with dynamic loadable driver
> modules.
>      This is reasonably doable, but is this acceptable for the bhyve
> maintainers?
>
>      For building the port, the bhyve-blockrbd code will only need a
> limited set
>      of files from /usr/src/usr.bin/bhyve thus limiting the chance of
> running out
>      sequence with the bhyve from base.
>
> Looking over these 4 options, I think that 4 is the most desirable one?
> But 2 would parhaps be workable for users as well, but the project might
> think
> otherwise.
>
> Are there other options?
> And/or is 4 the best way to go, with 2 as a nice intermediate?
>
> Thanx,
> --WjW
>

Great work!  I also agree that option 4 sounds like the best.  There's
precedent for ports that require the FreeBSD Sources.  For example, see
devel/py-libzfs or emulators/virtualbox-ose.  You just need to define the
SRC_BASE variable.