Re: bhyve disk performance issue

Hi,


> On 23 Feb 2024, at 18:37, Matthew Grooms <mgrooms@shrew.net> wrote:
> 
>> ...
> The problem occurs when an image file is used on either ZFS or UFS. The problem also occurs when the virtual disk is backed by a raw disk partition or a ZVOL. This issue isn't related to a specific underlying filesystem.
> 

Do I understand right, you ran testing inside VM inside guest VM  on ext4 filesystem? If so you should be aware about additional overhead in comparison when you were running tests on the hosts.

I would suggest to run fio (or even dd) on raw disk device inside VM, i.e. without filesystem at all.  Just do not forget do “echo 3 > /proc/sys/vm/drop_caches” in Linux Guest VM before you run tests. 

Could you also give more information about:

 1. What results did you get (decode bonnie++ output)?
 2. What results expecting?
 3. VM configuration, virtio-blk disk size, etc.
 4. Full command for tests (including size of test-set), bhyve, etc.
 5. Did you pass virtio-blk as 512 or 4K ? If 512, probably you should try 4K.
 6. Linux has several read-ahead options for IO schedule, and it could be related too.

Additionally could also you play with “sync=disabled” volume/zvol option? Of course it is only for write testing.

——
Vitaliy