From nobody Mon Oct 09 10:03:32 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4S3vkK5ZsCz4xGSj for ; Mon, 9 Oct 2023 10:03:37 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4S3vkK55QZz3cbf for ; Mon, 9 Oct 2023 10:03:37 +0000 (UTC) (envelope-from matthew@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1696845817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DI1Z5bvNLcgXAMHzdFFC9Mn+P6pOL/IDWR9Zw6W/39Q=; b=BoPftJ9Xa04psQbVIHJ5VzsE1YgID01VMSBwPp8IhvktBdEYUzrhFmwQBrKHnlBr4PrK0G yMqmEVAStq4Kle7lXuXFGq4SbsOLKxufXFRF0dBlCu1riPLtxxMQBO3E6CmA5bDFFyc5tX m6Sy8JDU/yZYH/fhiRiuHVBa2tI2NQrycVP9FrDCQfugdh0Y3KgKlHH5PlN1PlP6EkrDQH HHCQbr3QaKUgH/p0JQfGnmLTTWNGHGt/iN+YucvCKvufAhAZejkFOQtdcngddHVPvEWWbX Wbc5PNAk4yGwDRObv3bEqgrhLmEjJ2qKUqjRmZb8llGPwFaof51EL9+bG1AnUA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1696845817; a=rsa-sha256; cv=none; b=T2J4RdjwRZCZ04y2Uy3Z6wEWC8iIKrlz8qDj+e+WGxJLifgW4UMAtYcypc/6pkEIfk9ObX jLltmtICv4mDb1yXUCzCt1ufkxktqA7cG1GSzxvl1ntE9/QZI1o3MsXb1SxLgufzb31U3H V8gALtzJQM6SPxio3fsyPAjTa0xkMw+wMIH470b4I0yvoUE5i+3562BnoQ6PSlaFcsz0Pu yfgRNHIZP7VbMobM8vfsj9B1OOSV5+I1jclBF0LylcpTt7R/p6ZO+dcUU8PsfqopDXGdjd ut2AgqgeHfkNoNT8lbh4aZeIdWjUh6enWdMO7xKKDjc/oT8+1SrGDcsr5NTktg== ARC-Authentication-Results: i=1; smtp.infracaninophile.co.uk; dmarc=none (p=none dis=none) header.from=FreeBSD.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1696845817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DI1Z5bvNLcgXAMHzdFFC9Mn+P6pOL/IDWR9Zw6W/39Q=; b=tZ7TL76ncjP/Z8NUh0URCZObInudxPvcEy0B4NlSvzDaJ3hxhF+PhMYsxza+CnuyoczKIQ DYZV+YFIp/jjtTZeiAwqT00v0u/WbK8KocoPeRbS683YtAC0bkwufBqFZy4gsJ5/GzBqW7 HKcr4+hRVvtAOHskasSodmQlXV1En+E8C1zoAXQ5sX8e0cf6p1wGS2PS1MPOMAjw8G1RW+ 2x4PJYj7GzNIOvlvfPF+k/ZBjx9O2UAlHprJ7aB1y9wTcbPPcDhepVNjZg2mPJbpREZUJ/ JmKIr7NBXM8oXrUaREwub2W7yIi7ZhdBv3hTuO/qoFn0KZG5edruyYhEjHPrIg== Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [81.2.117.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: matthew/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4S3vkK3GgKz7h4 for ; Mon, 9 Oct 2023 10:03:37 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from [10.8.0.14] (cantor-hypercube.oucs.ox.ac.uk [163.1.67.131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id BE7FB196EC for ; Mon, 9 Oct 2023 11:03:33 +0100 (BST) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none (p=none dis=none) header.from=FreeBSD.org Message-ID: <7dc49a50-fe6b-4975-b2a1-03a217a7e2e1@FreeBSD.org> Date: Mon, 9 Oct 2023 11:03:32 +0100 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: arc_prune thread consuming 100% CPU Content-Language: en-GB To: freebsd-fs@freebsd.org References: From: Matthew Seaman In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 06/10/2023 01:54, Mark Johnston wrote: > Sometime in the past two months I started seeing a problem with ZFS on > the latest FreeBSD main on my desktop. The symptom is that the arc_prune > thread, which attempts to shrink the vnode cache, consumes 100% CPU even > when the system is idle. I've been seeing this since around the beginning of September on STABLE/13 -- right after I updated because of FreeBSD-SA-23:10.pf etc. This doesn't affect me while the system is idle, but anything that involves some amount of disk IP triggers the effect: that can be as simple as running `git status` in a checked out ports repository. Git launches about 20 -- 30 threads, arc_prune immediately pegs one of the two CPUs to 100% usage, the system load goes to about 10 and more likely than not my PPP link drops. This is a fairly old Core2 Duo machine with 8GB RAM, which is pretty feeble by modern standards, but it's been fine under the sort of loads I've put on it until recently. Cheers, Matthew