From nobody Fri Dec 20 13:46:18 2024 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YF7xC4BgNz5MZJW for ; Fri, 20 Dec 2024 13:46:23 +0000 (UTC) (envelope-from infoomatic@gmx.at) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "Telekom Security ServerID OV Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YF7xB3CJ9z4gKv for ; Fri, 20 Dec 2024 13:46:22 +0000 (UTC) (envelope-from infoomatic@gmx.at) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.at header.s=s31663417 header.b=UDgbGBwz; spf=pass (mx1.freebsd.org: domain of infoomatic@gmx.at designates 212.227.17.22 as permitted sender) smtp.mailfrom=infoomatic@gmx.at; dmarc=pass (policy=quarantine) header.from=gmx.at DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmx.at; s=s31663417; t=1734702379; x=1735307179; i=infoomatic@gmx.at; bh=zilrQ3AYS2Syy+oE8kRCvFT80OLeMIjjU0HPvY28eqI=; h=X-UI-Sender-Class:Message-ID:Date:MIME-Version:Subject:To: References:From:In-Reply-To:Content-Type: Content-Transfer-Encoding:cc:content-transfer-encoding: content-type:date:from:message-id:mime-version:reply-to:subject: to; b=UDgbGBwzRyQ5R8wuewhbUkpxvQZHAwGDh7gnxLTxX/xlvwBZ7O7bKRYZpXhySoGR vgNbP3SZJg9JmA3Cq7SaMikXjQaqxLdIom1WShPc2QrWKy19kpDu/35hS/1QGWJfP Oro2ZcTYzfzeaKProzHKKXX+KCbaYEFW83nzJt8vPDcm7MPuk9Z2E+5Kn1p3PP3gv 7gb9NPQadVIleW4zcrMELiJAF8ZiCbIbPV3MolT4BLUD+H93efxjWdQvPaktnIm92 kabksoA0VhL8cjSzw6UvuFf4AmrhC0m8aMMOSv0LYC5xvoXqAM4n9JiVEVURSF6vc 2NsXy9UQLhz8MZl9jA== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from [10.0.1.209] ([178.114.190.115]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MxUnp-1tmcIk1t4D-00yjzV for ; Fri, 20 Dec 2024 14:46:19 +0100 Message-ID: Date: Fri, 20 Dec 2024 14:46:18 +0100 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-questions@freebsd.org Sender: owner-freebsd-questions@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bhyve performance issues To: questions@freebsd.org References: <3750c2b1-0621-4b49-b485-f4f6ec3e78bf@gmx.at> Content-Language: en-US From: infoomatic In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:dPUoDVw/Gp2RvK363Yf4WKaWsJJfPwPwDF4IzuQlvy+/SP2BPu8 QZV6N7t76He6GMb7k+Gp3C855WxdewIjhWxlNuW2pl/gyrnz/UrBMTDcqLckyy590ZuJdGI T1GwunCoH4/FkCh1Dbo0FGHTUy1nmTl03m2IaCzzvjvo+f0hxes1ErU56miHrBTYtDFXR8x yQwJUL25iuPDybafJyyww== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:AzLsPY19WIk=;jMRVI8PxK9Ad+3qhpKGS5Mai6ao 5mi4xExqTEGfUedk4SW2adJ9rZHBYBivGzkLzz6LcOez2OjMPfql0w5iDB4di/kkUkNyslP5Z xgU8aBaZ1xDVES2G5zTyRx7jra7FEhZRJY/O+I4iHbJT6qHTGuS0/Hy8ped9uF4v7CrEnr4n8 /g3U8sCgbR4ny83p5NTzkwvrHcsU6r+kvTzC8g5HbtrUl56B869aZJoXkG4/36UM9qxjsDJTt 4D1RIk/p1ej9HvR+W3N2zJbUSrPo7juTBX/gQQzzWcfdnR8uuBrm/r18bjcHujvB7LxYb1UBy W2Te2iolJ5LcO7mmNWCWl6nOFoF31wDavhlziB1TXbUfqLbl3Gj2i8zsGK3vb/9J9/9v+gcGR Q23MAWpw/p5G+ZGL4ttTtIsyD0uFPey+06DTu+BpBOVXzws1Ys36KOloRTuq/yl8j+Y9dNETB dXkVfMksEs3M8eu1lnnqvJBIG2KLhLQUtN+Z/4zvA1zdnQFjohnplBhctmSXdRFvJl3E0fY9K Z6poYs3YwO5LAEt0SvkvmxLLcdLDz2iGknKRAi6d9zkF+gSg5twZInfe1gBIUGryveiOMSMe9 3VZ6M1jTMmdu9Mquk1BODZp1txzwOp8fS1A+Wa6oH/OFhmhs14vnlQ3UnKV8RR+ZleddtBpbN Bisj58qixjPbKssERufSvz+evew1jQrfGrNgP3RsChEE7+4bH2fjeRZvwduze167qhT94vrOh nf6t5tkT51lJuYt5z/iKh3Y2CSSULBGd3G7Jd8OosA9xP4qpD5afumXtKAG+GUznEjDLcrEfj TvWGBH3iT+ycBtAOMART6QxN26k0OmFwN8iumjsL/j7xSUC8YWlKs+yT63tJNu9aVice5x+67 E1ESeM2x0Mzut0SuYdm0c+8suhAEi2y9/kbNTl/oUjhonzDLu0aEeZthbAl9FKX7m8qWdZmgG UB7b75MBVm2KKH/ikdbpN4nqR2G4V25SssUXBRuDmppTGbDQEzlta7lYmvBbChYHUsRe0ovsl RYdpUGXKbQy6FYiHaa/GcIbMGvWNaspkS8EgEbtIsgfXRE6p1W0uM3Ebg+jQYxYQ5rEi7OmVp Eqmr3+5oBeVNVK+hmeaXc0wIatRM0k X-Spamd-Result: default: False [-3.89 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_LONG(-0.97)[-0.970]; NEURAL_HAM_SHORT(-0.73)[-0.734]; DMARC_POLICY_ALLOW(-0.50)[gmx.at,quarantine]; R_DKIM_ALLOW(-0.20)[gmx.at:s=s31663417]; RWL_MAILSPIKE_VERYGOOD(-0.20)[212.227.17.22:from]; R_SPF_ALLOW(-0.20)[+a:mout.gmx.net]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.17.22:from]; XM_UA_NO_VERSION(0.01)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[gmx.at:+]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_FROM(0.00)[gmx.at]; PREVIOUSLY_DELIVERED(0.00)[questions@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[questions@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmx.at] X-Rspamd-Queue-Id: 4YF7xB3CJ9z4gKv X-Spamd-Bar: --- On 20.12.24 07:25, Daniel Tameling wrote: > htop is not 100% reliable. You should verify it with top. > > top -S show you system processes. That should tell you what makes the sy= stem unresponsive. You could also check use systat. Pure systat shows you = the system load and systat -iostat shows you whether the linux build maxes= out the IO capabilities of your system. > This was my initial suspicion, too. However, it was roughly the same as top -S has shown, processes with with most WCPU are bhyve (~150%) and kernel and idle jumping back and forth between 5 and 500% and here and there zfs appearing, so nothing unusual. systat -iostat shows IO is pretty saturated, and no weird interrupt storms or something unusual. When the system is in this unfavorable state, a DNS lookup using drill on the host takes > 10 seconds (the host is in a datacenter and has a decent 1GBit connection, usually this is done in under 100ms (mostly between 15 and 50)) I found this [1] thread and tried to replace virtio-blk with nvme. This improved the situation - reducing load to 20-25 (dns request on the host ~ 4 seconds). Build time in the linux vm is down to 17 minutes from 23. Further improvement can be reached by setting priority=3D"10" in the vm.conf file which increases niceness/lowers priority. Build time increases by roughly 20 seconds (which I really do not care in this setting), whereas responsiveness on the host increases quite a bit. Using priority=3D"16" adds just 10 seconds more and further increases responsiveness to a level where working on the host is fine and one barely recognizes that the system is actually doing something in a VM, which is my desired state. I will also have a look at the rctl settings and maybe do some scheduler tweaking - the sysctls kern.sched come to mind. However, imho a guest being able to cannibalize the hosts resources in such a way that the host becomes barely usable (with the default configuration) should be improved. Thanks for your input! Robert [1] https://lists.freebsd.org/archives/freebsd-virtualization/2024-February/00= 1952.html