From nobody Wed Dec 01 18:13:48 2021 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C797E18C2311 for ; Wed, 1 Dec 2021 18:14:00 +0000 (UTC) (envelope-from olevole@olevole.ru) Received: from mail-oi1-x232.google.com (mail-oi1-x232.google.com [IPv6:2607:f8b0:4864:20::232]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J46fc4pSbz3GBY for ; Wed, 1 Dec 2021 18:14:00 +0000 (UTC) (envelope-from olevole@olevole.ru) Received: by mail-oi1-x232.google.com with SMTP id t19so50289724oij.1 for ; Wed, 01 Dec 2021 10:14:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=olevole-ru.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=UlFAiDZA20RM7atm9SZUx9vTOGU12B7CmO66meCQPH0=; b=q6VlzkmRmh0aoZEIGyx4VQPy5yEbjlF3R3DC6CmUZuqqrKOzEnKAu1Z3ohNo41UCw+ 4OA0cqAWnPeV0GFOcwDvudZ2luray785CFq4S2K6I1pxaV+9fTkRonoWuBEhwMk+RUbP ChOZXxPq1Xr8yxtbMO0PQwm40pkCgq5SC00FKCKPNVVQzLd+lymj25AY/lqMG9K46vzp jugzKW69cilfIxMVB9tAhklRZrMXWzRINOBubHrLcIt+iSwbvNURTD0RzVYDxdH34s7i lrFF8abohB1GOlppJzIkoEwonWq4DB69KCvweiQt5bizFzurYqCqIaVGhRcklU6TiIqj 4v5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=UlFAiDZA20RM7atm9SZUx9vTOGU12B7CmO66meCQPH0=; b=j9LFpD9/AVb72nz6TLHwZ6G3884c3K9wexl/WtEMHCmY4LIlVT6Z5H7fFYMKRxQEHM NhmOBCviJSyrIEYBFTRInzpw0eq9a5qPvuScndlo2EVMsFKoxrdK0gt4LvhYUiuI1Bzc YFUlr1VYqTKVgMnk1vSQvoKMHndOD+ZFJVERlmZ0gULcau3SfjPcdOpUASKN5D0ZjkpW feji3q0V8fU95pD1sNTtHp1gTC/g7nmBxyUiV693g4j9GT1QutfM4Nzd7xEdjcV1DZ3m jYABRtc+2oheTDSOV9aVSPR8bMlh/eO7/Kz1PpGb6RTN4VI1f/IdBg+bmzOQ7MlZqQgx m14A== X-Gm-Message-State: AOAM533zeSAOmdZZux446tIpVnRogWC211h3KN6DxAOxS7V9BVux6YlY +kXgpQ7kJg+dQYg13TwkZOKZbwPZgYjmpWit1PI82Q== X-Google-Smtp-Source: ABdhPJyR6FpKmA1HB0jtTrmnNXZBS0+Xlmevf8XPfm6vIuNHqQPbRpwgLqeaOi77f7Tb3i2Aqp8PZY8z44nbbMQn/F8= X-Received: by 2002:a05:6808:23d6:: with SMTP id bq22mr7654802oib.71.1638382439879; Wed, 01 Dec 2021 10:13:59 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: <4E8A7FD3-B01E-4ADE-A290-360F3B04AC0F@jld3.net> <30e4454c-414a-833f-3829-586a450e7205@quip.cz> In-Reply-To: <30e4454c-414a-833f-3829-586a450e7205@quip.cz> From: Oleg Ginzburg Date: Wed, 1 Dec 2021 21:13:48 +0300 Message-ID: Subject: Re: bhyve vCPU limit To: Miroslav Lachman <000.fbsd@quip.cz>, rgrimes@freebsd.org Cc: bsdlists@jld3.net, jbo@insane.engineer, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4J46fc4pSbz3GBY X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Hi On Wed, Dec 1, 2021 at 7:52 PM Miroslav Lachman <000.fbsd@quip.cz> wrote: >I was playing with VMs in VirtualBox and Bhyve and compared > performance with increasing vCPU count. The more cores VM get the slower How many physical cores you have on the host? This is a characteristic behavior when overcoming. But without overcommit I observe productivity growth: 1) Single vCore benchmark via cpuminer: https://pastebin.com/mg46RvDT TOP from host: https://pastebin.com/LvbBiyFz 2) Sixteen vCore benchmark via cpuminer: https://pastebin.com/7FQjaVM8 TOP from host: https://pastebin.com/dLpreq9D I observe almost linear growth. In addition, the loss of performance can be associated with NUMA (required cpuset/cpuset_setaffinity and control of memory capacity) As far as I remember, it is not enough to simply change one parameter in .h file: https://bhyvecon.org/bhyveconOttawa2019-Rodney.pdf