Re: bhyve disk performance issue

From: Matthew Grooms <mgrooms_at_shrew.net>
Date: Sat, 17 Feb 2024 21:53:10 UTC
On 2/16/24 12:00, Matthew Grooms wrote:
> On 2/16/24 11:42, Chuck Tuffli wrote:
>> On Fri, Feb 16, 2024, at 9:19 AM, Matthew Grooms wrote:
>>>
>>> Hi All,
>>>
>>>
>>> I'm in the middle of a project that involves building out a handful 
>>> of servers to host virtual Linux instances. Part of that includes 
>>> testing bhyve to see how it performs. The intent is to compare host 
>>> storage options such as raw vs zvol block devices and ufs vs zfs 
>>> disk images using hardware raid vs zfs managed disks. It would also 
>>> involve
>>>
>>>
>> …
>>>
>>> Here is a list of a few other things I'd like to try:
>>>
>>>
>>> 1) Wiring guest memory ( unlikely as it's 32G of 256G )
>>> 2) Downgrading the host to 13.2-RELEASE
>>
>> FWIW we recently did a similar exercise and saw significant 
>> performance differences on ZFS backed disk images when comparing 14.0 
>> and 13.2. We didn’t have time to root cause the difference, so it 
>> could simply be some tuning difference needed for 14.
>
> Hi Chuck,
>
> That's very helpful feedback. I'll start by downgrading the host to 
> 13.2 and report back here.
>

Unfortunately same story with 13.2. I'm going to try and downgrade to 
12.4 and see if it gets any better.

================================================================================
Begin @ Sat Feb 17 11:00:01 CST 2024

Version  2.00       ------Sequential Output------ --Sequential Input- 
--Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
localhost.lo 63640M  690k  99  1.5g  97  727m  78  950k  99  1.3g 68 
+++++ +++
Latency             11759us   29114us    8098us    8649us 23413us    4540us
Version  2.00       ------Sequential Create------ --------Random 
Create--------
localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
                  16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ 
+++++ +++
Latency              7791us     131us    1671us     464us 15us    1811us

End @ Sat Feb 17 11:03:13 CST 2024
================================================================================
Begin @ Sat Feb 17 11:10:01 CST 2024

Version  2.00       ------Sequential Output------ --Sequential Input- 
--Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
localhost.lo 63640M  667k  99  449m  99  313m  94  940k  99  398m 99 
16204 563
Latency             12147us    1079us   24470us    8795us 17853us    4384us
Version  2.00       ------Sequential Create------ --------Random 
Create--------
localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
                  16     0  93 +++++ +++ +++++ +++     0  96 +++++ +++ 
+++++ +++
Latency               118us     159us     269us     164us 54us     844us

End @ Sat Feb 17 11:18:43 CST 2024