bhyve: zvols for guest disk - yes or no?

[Volodymyr Kostyrko]

Jan Bramkamp wrote:
> An other thing I learned the hard way is that ZVOL are set in stone at
> the ZVOL creation. You have to (cam)dd everything to change the block
> size. The default ZVOL block size is 8K which isn't wrong but your
> guests need to align their file systems (and swap) correctly or you'll
> suffer from write amplification. And ZFS RAID-Z really sucks for such
> small block sizes. Use mirrored VDEVs in your pools or you will suffer
> from massive metadata overhead and disappointing IOPS.

This pole has two ends though.

When you are working with file system default 8k block size is too 
small. Setting it up to 64k will save a lot writes for host. This is due 
to most current filesystems do work correctly with extents/big blocks:

  * Linux: ext4, xfs - full support;
  * BSD: ufs - disabled, need to be enabled on format, zfs - full support;
  * NTFS: no support, but you can use 64k blocks with Win XP - Win 10 
(though Win XP can't boot off such partition).

Setting block size bigger makes fragmentation less common (check your 
`zpool list`) and saves writes. When guest writes one 64k chunk ZFS 
writes 8 separate 8k blocks (+ metadata and stuff) and this is not good 
for the speed.

On the opposite when you have some database inside VM you need to 
prepare disk for it accordingly. Guides for using PostgreSQL and MySQL 
apply to VM's too. MSSQL on the other hand uses 64k extents to work with 
database internally so raising block size to 64k would be good for it.

-- 
Sphinx of black quartz judge my vow.