From nobody Sat Jan 29 01:11:11 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 520DB19907E1; Sat, 29 Jan 2022 01:11:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jlx9D186jz4kjM; Sat, 29 Jan 2022 01:11:12 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643418672; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sf8DBqUm6IxM3y9QMvqOFy/i9zcto0wyhE8MyJFjLUU=; b=lLBXY+MoGoR3fyMMhWiQLNNlX2ndUvEvv2Lih+4sfdeadXLtj8UG/jtIWsMN+f8amszWW2 rZVVm2jUzBADXp8QPSLYZcauSTfZ/TmSrBXDxMEx88jD/XT8TcXvlGd4z08OwtNgcX6pVs n+dxINKmtAsSE/vUdibS6k8pqnO9zIHUd0w9lWC5kolFKZanhptdUE9bo3nr3JbcAgiKpm FGW5LwBlKVXKFC+vqF3Wc188WeTbUkevPMaKa2LphZEKzE9Nzx15Suk5CzWXEuvcGu8JNj 9ivL+X6yLarMYNXUx9yYZ3bBoE4C+4cxpQV/f1CjKQwqcnK6n7g2TmCfaRCSJw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 02CC544EA; Sat, 29 Jan 2022 01:11:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20T1BBHx039397; Sat, 29 Jan 2022 01:11:11 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20T1BBRL039396; Sat, 29 Jan 2022 01:11:11 GMT (envelope-from git) Date: Sat, 29 Jan 2022 01:11:11 GMT Message-Id: <202201290111.20T1BBRL039396@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: 2366d7ce8716 - stable/13 - vm_pageout_scans: correct detection of active object List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 2366d7ce87164b862e278a80ec896f5afcb5dbd5 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643418672; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sf8DBqUm6IxM3y9QMvqOFy/i9zcto0wyhE8MyJFjLUU=; b=AVOsq67lRYkKQI0iMSbyEJxDFJ0Lgml+I764U2rfui4ELagrHULNatwnI1dHD+jQRjRMlm vD3Dlz8ik/PoCdK1zj9j9K8Ca2UxgC6zyT5YvKCJpVeyipyTpjbNmoj+pNhvSULl0vDfUf TOc8CUDSNKnxtqYcGGv4mAi5Fs7bAeoMHWu0ZmJvQmbJEfWcM07daoj/RV8Uz2FoVidyIE SRxvwsi9OoAp1lRWfDo8bsHX1Vqt5Xnh01sF/yRA3fmyGR+O2C8xdF6/vNULNELaTPYjPk m2rFJ/COxPr4gOeyM6agjECoXW05CgQsUIg6VF1HekQoRFQsyam4xUNiwxpSyQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643418672; a=rsa-sha256; cv=none; b=IO/KW7lhVlpRiORHX67fpPgPrdfvBMGPpjFTmY3q/ZXoL/FJW5kNgDCZCmgeMjKd6fafsH xmQLxS2h7vEhlQBLD0Gg7NggkEa7FjzUjHarTkxgGPG6s7ZIhDXKQEY9YoJI8y7m4SwUGM Bm7Aw7SHoeEZsoCXI9r3JjV5q1Sfl3FtdhkJPdG8pJPp83PbRHfecgAlSLcZIpQWVxq8DR bg+QNKotmJ93XASRLNVIS70edYdeA68tANeT9RQZAfT+MwoOg98ru6nAdtMHUN5v0TIJ9r 58ASqI1ElsuqDLD6Uzc8MlPZOgzO3I+XgUhsWwsAH2GVkd2Cw8CmkI/Gsppv6w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=2366d7ce87164b862e278a80ec896f5afcb5dbd5 commit 2366d7ce87164b862e278a80ec896f5afcb5dbd5 Author: Konstantin Belousov AuthorDate: 2022-01-16 20:18:21 +0000 Commit: Konstantin Belousov CommitDate: 2022-01-29 01:10:44 +0000 vm_pageout_scans: correct detection of active object (cherry picked from commit 3de96d664aaaf8e3fb1ca4fc4bd864d2cf734b24) --- sys/vm/vm_pageout.c | 56 ++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 18 deletions(-) diff --git a/sys/vm/vm_pageout.c b/sys/vm/vm_pageout.c index 899d35cd43b6..64a06dd7d55b 100644 --- a/sys/vm/vm_pageout.c +++ b/sys/vm/vm_pageout.c @@ -712,6 +712,38 @@ unlock_mp: return (error); } +/* + * Check if the object is active. Non-anonymous swap objects are + * always referenced by the owner, for them require ref_count > 1 in + * order to ignore the ownership ref. + * + * Perform an unsynchronized object ref count check. While + * the page lock ensures that the page is not reallocated to + * another object, in particular, one with unmanaged mappings + * that cannot support pmap_ts_referenced(), two races are, + * nonetheless, possible: + * 1) The count was transitioning to zero, but we saw a non- + * zero value. pmap_ts_referenced() will return zero + * because the page is not mapped. + * 2) The count was transitioning to one, but we saw zero. + * This race delays the detection of a new reference. At + * worst, we will deactivate and reactivate the page. + */ +static bool +vm_pageout_object_act(vm_object_t object) +{ + return (object->ref_count > + ((object->flags & (OBJ_SWAP | OBJ_ANON)) == OBJ_SWAP ? 1 : 0)); +} + +static int +vm_pageout_page_ts_referenced(vm_object_t object, vm_page_t m) +{ + if (!vm_pageout_object_act(object)) + return (0); + return (pmap_ts_referenced(m)); +} + /* * Attempt to launder the specified number of pages. * @@ -806,7 +838,7 @@ scan: if (vm_page_none_valid(m)) goto free_page; - refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; + refs = vm_pageout_page_ts_referenced(object, m); for (old = vm_page_astate_load(m);;) { /* @@ -826,7 +858,7 @@ scan: } if (act_delta == 0) { ; - } else if (object->ref_count != 0) { + } else if (vm_pageout_object_act(object)) { /* * Increase the activation count if the page was * referenced while in the laundry queue. This @@ -1263,20 +1295,8 @@ act_scan: * Test PGA_REFERENCED after calling pmap_ts_referenced() so * that a reference from a concurrently destroyed mapping is * observed here and now. - * - * Perform an unsynchronized object ref count check. While - * the page lock ensures that the page is not reallocated to - * another object, in particular, one with unmanaged mappings - * that cannot support pmap_ts_referenced(), two races are, - * nonetheless, possible: - * 1) The count was transitioning to zero, but we saw a non- - * zero value. pmap_ts_referenced() will return zero - * because the page is not mapped. - * 2) The count was transitioning to one, but we saw zero. - * This race delays the detection of a new reference. At - * worst, we will deactivate and reactivate the page. */ - refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; + refs = vm_pageout_page_ts_referenced(object, m); old = vm_page_astate_load(m); do { @@ -1526,7 +1546,7 @@ vm_pageout_scan_inactive(struct vm_domain *vmd, int page_shortage) if (vm_page_none_valid(m)) goto free_page; - refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; + refs = vm_pageout_page_ts_referenced(object, m); for (old = vm_page_astate_load(m);;) { /* @@ -1546,7 +1566,7 @@ vm_pageout_scan_inactive(struct vm_domain *vmd, int page_shortage) } if (act_delta == 0) { ; - } else if (object->ref_count != 0) { + } else if (vm_pageout_object_act(object)) { /* * Increase the activation count if the * page was referenced while in the @@ -1584,7 +1604,7 @@ vm_pageout_scan_inactive(struct vm_domain *vmd, int page_shortage) * mappings allow write access, then the page may still be * modified until the last of those mappings are removed. */ - if (object->ref_count != 0) { + if (vm_pageout_object_act(object)) { vm_page_test_dirty(m); if (m->dirty == 0 && !vm_page_try_remove_all(m)) goto skip_page;