UFS vs. ZFS inside bhyve hosted on ZFS

[Russell L. Carter]

On 03/18/16 18:48, Russell L. Carter wrote:
> Greetings,
>
> So I am becoming quite enamored of an 11-current bhyve guest installed
> with ZFS root running on a 10-stable host with ZFS and driven by an
> AMD FX-8320.  However the 3 cpus I give it seem to translate to a lot
> of overhead on the host when building the -current world, determined
> by drinking beer and watching htop on both.  The guest is running a
> GENERIC-NODEBUG kernel, and I'm building with MALLOC_PRODUCTION=yes.
> The host is much leaner.
>
> So I am wondering if UFS in the -current guest might be better
> overall.  I can certainly do a multiple hour experiment, installing
> a new guest with UFS root, but since I am new to this, perhaps there is
> conventional wisdom about ZFS vs. UFS in the guest?  Maybe UFS in the
> guest requires less cpu resources from the host?  Or not?

I had an off-list suggestion to go ahead and DO THE WORK, so I'm now
fairly far down the path.  I have two bhyve VMs configured
identically, both running r297047, each configured with 15G of RAM and
3 cpus, verified by examining 'sysctl hw' from within the running
guest.  One has the default ZFS install, the other UFS.  However,

# cd /usr/src && make buildworld -j <anything>

after 15 minutes or so freezes the -current-UFS guest with the
following console output:

root at vm1:~ # ahcich0: Timeout on slot 29 port 0
ahcich0: is 00000008 cs 00000000 ss 00000000 rs bfffffff tfd 50 serr 
00000000 cmd 0001dd17
(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 70 dc 3a 40 00 00 
00 00 00 00
(ada0:ahcich0:0:0:0): CAM status: Command timeout
(ada0:ahcich0:0:0:0): Retrying command

I'm using the defaults for the iohyve script, which are easily
extracted if required.  The -current-ZFS guest has been completely
solid, building ports and world/kernel across multiple revisions.
That said I am completely new to this so likely have misconfigured
something.

Thanks,
Russell