From nobody Wed Jul 10 22:05:42 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WKBkZ4W6Kz5RFNg; Wed, 10 Jul 2024 22:05:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WKBkZ3d6zz4KSZ; Wed, 10 Jul 2024 22:05:42 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1720649142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=23TFtYWxKi8SKnhs6L3S7OUYnL6zfdJ18Kq58myRgTA=; b=tmvzdKzbWlhGNK6OBfuYMNjosK3s/xuzjMQucP2YmpZ/MuV1OdeuWziofcCr8imtPtzLbb CURSBiknCoWFpiZa2jQro4IPbMuAeJ/090QxF5Fct2dHC+5KVEaL3UFUs9SpSBphTveWB1 Q8bbrPv5Xn/UF1xA/0MYg2f5siX2MerP+wOs2pZB35E3LZXFYpNI62q5hWPqUxH7/btNia KouI1q//HwcJQ4S/7uTkCj7FIMF1CW+qEzaXfNdjDYPIrIATUnZxb0ciE+/kjyFGu5h6kr lSFa8WZwBCAYlGxw/KytMpPyJYEpzsRhVN7QA0ddgwDWp2mwkC9JcWlzpAv5Nw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1720649142; a=rsa-sha256; cv=none; b=n7/pVhYztH4MeQQo1lEhS/XOR10u0WX1uQ37CjI5PkJlgnsQIh50Lq37eXYPtJHkAqFwii Tsc0JOWJx8BZy7WpmV8fOvc21nLKHT59HZa1/zg5cFd6Be+4MsJkgWJr5k9M4o+/FTpJq4 nVTpO4NQrbvmb21APaEtYFTH6ZPYaUBI04Xvo74cI0l91kWjgWiE6XTf6Oj4iiwB/UE3eu 4+N0je5M3WK3woVI2RtVhnYtCxeG27QpyOmhiMCP/O+Mj0MrIjfUpvC3+fZmoTZ8qDM79i Xu0gj85oQWrsYeZa8M+ma/TrMU7PcNOZuERGbm6UyZtUu4juhyG4o46JHh/pdw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1720649142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=23TFtYWxKi8SKnhs6L3S7OUYnL6zfdJ18Kq58myRgTA=; b=RX/TBiboG9xyXJ76ULoZO46rrxuj+2JkY49Sen+cXLCvIgNdIBZ3CqHkN9m2SCtgIX7QK5 V11fXz/D9d2RJVPMoDUVgZzV8DKA/dZmk/eQzaETV9EfFY5xuXNpSyGW3idrQeYk+MWpK4 6K6sVfP0zEv9Fe2pyhqbzBtEm9/pOmqtdYNj+D9OnH8uH7wH0m06noO5lPFtvytNktc218 TxIoZVoTiqBOtSWI86gRNYaSVxkxl04mqxgVe+C3YwMS2EXrc3wlVsrC/iVXWxvMpNNnW1 MOvSfZO4Q3nwWSfWLDNhdgtOsCVoXSacMYGi/6gSlbViZxx+nzHFYiQgFn/wBQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WKBkZ3CzwzZtN; Wed, 10 Jul 2024 22:05:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46AM5g5A079901; Wed, 10 Jul 2024 22:05:42 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46AM5goS079898; Wed, 10 Jul 2024 22:05:42 GMT (envelope-from git) Date: Wed, 10 Jul 2024 22:05:42 GMT Message-Id: <202407102205.46AM5goS079898@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: b7f6841e00d5 - stable/14 - vfs: make skipping LRU requeue optional List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: b7f6841e00d53f1aee65a8cce4f98c239ae4cf75 Auto-Submitted: auto-generated The branch stable/14 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=b7f6841e00d53f1aee65a8cce4f98c239ae4cf75 commit b7f6841e00d53f1aee65a8cce4f98c239ae4cf75 Author: Mateusz Guzik AuthorDate: 2024-07-08 12:24:41 +0000 Commit: Mateusz Guzik CommitDate: 2024-07-10 22:04:40 +0000 vfs: make skipping LRU requeue optional As explained in the comment in the code it is a bottleneck in certain workloads. On the other hand it does not need to be skipped in most cases, while transiently running into the lock being contended happens a lot. (cherry picked from commit 0a9aa6fdf58468945240e86bf16c268acc8c1776) --- sys/kern/vfs_subr.c | 54 +++++++++++++++++++++++++++++++++-------------------- 1 file changed, 34 insertions(+), 20 deletions(-) diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index d1c17dca37d4..646339987ba2 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -224,6 +224,10 @@ static counter_u64_t vnode_skipped_requeues; SYSCTL_COUNTER_U64(_vfs_vnode_stats, OID_AUTO, skipped_requeues, CTLFLAG_RD, &vnode_skipped_requeues, "Number of times LRU requeue was skipped due to lock contention"); +static __read_mostly bool vnode_can_skip_requeue; +SYSCTL_BOOL(_vfs_vnode_param, OID_AUTO, can_skip_requeue, CTLFLAG_RW, + &vnode_can_skip_requeue, 0, "Is LRU requeue skippable"); + static u_long deferred_inact; SYSCTL_ULONG(_vfs, OID_AUTO, deferred_inact, CTLFLAG_RD, &deferred_inact, 0, "Number of times inactive processing was deferred"); @@ -3785,31 +3789,41 @@ vdbatch_process(struct vdbatch *vd) * lock contention, where vnode_list_mtx becomes the primary bottleneck * if multiple CPUs get here (one real-world example is highly parallel * do-nothing make , which will stat *tons* of vnodes). Since it is - * quasi-LRU (read: not that great even if fully honoured) just dodge - * the problem. Parties which don't like it are welcome to implement - * something better. + * quasi-LRU (read: not that great even if fully honoured) provide an + * option to just dodge the problem. Parties which don't like it are + * welcome to implement something better. */ - critical_enter(); - if (mtx_trylock(&vnode_list_mtx)) { - for (i = 0; i < VDBATCH_SIZE; i++) { - vp = vd->tab[i]; - vd->tab[i] = NULL; - TAILQ_REMOVE(&vnode_list, vp, v_vnodelist); - TAILQ_INSERT_TAIL(&vnode_list, vp, v_vnodelist); - MPASS(vp->v_dbatchcpu != NOCPU); - vp->v_dbatchcpu = NOCPU; + if (vnode_can_skip_requeue) { + if (!mtx_trylock(&vnode_list_mtx)) { + counter_u64_add(vnode_skipped_requeues, 1); + critical_enter(); + for (i = 0; i < VDBATCH_SIZE; i++) { + vp = vd->tab[i]; + vd->tab[i] = NULL; + MPASS(vp->v_dbatchcpu != NOCPU); + vp->v_dbatchcpu = NOCPU; + } + vd->index = 0; + critical_exit(); + return; + } - mtx_unlock(&vnode_list_mtx); + /* fallthrough to locked processing */ } else { - counter_u64_add(vnode_skipped_requeues, 1); + mtx_lock(&vnode_list_mtx); + } - for (i = 0; i < VDBATCH_SIZE; i++) { - vp = vd->tab[i]; - vd->tab[i] = NULL; - MPASS(vp->v_dbatchcpu != NOCPU); - vp->v_dbatchcpu = NOCPU; - } + mtx_assert(&vnode_list_mtx, MA_OWNED); + critical_enter(); + for (i = 0; i < VDBATCH_SIZE; i++) { + vp = vd->tab[i]; + vd->tab[i] = NULL; + TAILQ_REMOVE(&vnode_list, vp, v_vnodelist); + TAILQ_INSERT_TAIL(&vnode_list, vp, v_vnodelist); + MPASS(vp->v_dbatchcpu != NOCPU); + vp->v_dbatchcpu = NOCPU; } + mtx_unlock(&vnode_list_mtx); vd->index = 0; critical_exit(); }