Re: bhyve vCPU limit

On 01/12/2021 19:13, Oleg Ginzburg wrote:
> Hi
> 
> On Wed, Dec 1, 2021 at 7:52 PM Miroslav Lachman <000.fbsd@quip.cz> wrote:
>> I was playing with VMs in VirtualBox and Bhyve and compared
>> performance with increasing vCPU count. The more cores VM get the slower
> 
> How many physical cores you have on the host? This is a characteristic
> behavior when overcoming.
> But without overcommit I observe productivity growth:

I tested it on 2 machines. One had 6 cores 12 threads, the second 4 
cores 8 threads. Even if the host machine was almost idle, only one VM 
was running the slowness with more than 1 vCPU was so significant. Let's 
say 0.01 sec for 1 vCPU, 0.3 s for 2 vCPU, 0.9 s for 4 vCPU. (on machine 
with real 4 cores / 8 threads).
With overcommit of vCPUs it was like 3 seconds with 6 vCPU and 8 seconds 
for 8 vCPU. Still the same task on the same idle machine.

On the other hand I can start 5 VMs with 2 vCPUs each and "everything 
seems normal" even if the total count of vCPUs are 10 on machine with 4 
physical / 8 threads.
In fact I tested with total count of 15 vCPUs distributed between 5+ VMs 
but the performance problem was always visible on VM with more than 2 vCPU.

> 1)
> Single vCore  benchmark via cpuminer: https://pastebin.com/mg46RvDT
> TOP from host: https://pastebin.com/LvbBiyFz
> 
> 
> 2)
> Sixteen vCore benchmark via cpuminer: https://pastebin.com/7FQjaVM8
> TOP from host: https://pastebin.com/dLpreq9D
> 
> I observe almost linear growth.
> 
> In addition, the loss of performance can be associated with NUMA
> (required cpuset/cpuset_setaffinity
> and control of memory capacity)
> 
> As far as I remember, it is not enough to simply change one parameter
> in .h file: https://bhyvecon.org/bhyveconOttawa2019-Rodney.pdf

Thank you for your data!

Kind regards
Miroslav Lachman