From nobody Sat Jan 22 17:35:56 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 48004196C659; Sat, 22 Jan 2022 17:35:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jh3Lj1Jb3z3tMZ; Sat, 22 Jan 2022 17:35:57 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642872957; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ch8lUOUtoDYdm3xVgYK90nppJvXs15l5YIjIyDd1mis=; b=t6M0KBaY7HYP9abG3snYXZjzxZthBOof8UQuHhJfm/eCV63GWI+iBVwhCt6MeJQ6yMAXqj Jtkn8egVi7HoTMX87MI4ggWG4PhblWWqmTWqHkv5rSnfQyPX6ew62b0QTpxETcbhfv1CZV ojeC+XPD9JGAGrGmYxnA5MFqSaWR0+inxna+5cuQLWIhC5udSUzGI2r9nRt3PP6tEEBDI4 JQpPFBBHjgA57nKyO+v5RB0t9DDR2zepj8u6d61mJGE5VNGOQrlb28TzirBdsj054009Zf bicVURyMFycN8/Trqx9LKLvxLTJQQelGsPlRxzdk5W0gN2nLmiJq3L4Ky2CHRQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0CD2D134A6; Sat, 22 Jan 2022 17:35:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20MHZuDK039858; Sat, 22 Jan 2022 17:35:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20MHZuq9039857; Sat, 22 Jan 2022 17:35:56 GMT (envelope-from git) Date: Sat, 22 Jan 2022 17:35:56 GMT Message-Id: <202201221735.20MHZuq9039857@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: 3de96d664aaa - main - vm_pageout_scans: correct detection of active object List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 3de96d664aaaf8e3fb1ca4fc4bd864d2cf734b24 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642872957; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ch8lUOUtoDYdm3xVgYK90nppJvXs15l5YIjIyDd1mis=; b=Q9u/1dfxcKU79K4F8O1dYmGw+2/+dWENeCPHj89mKFcfTCjpvLfPR5NZ4H0JRYzxWOjJTe lCEcea6545OgkDIxNlTFimk63GKskFRMfJ7fqwBtXqp4xJvIRLntHLkWIi6IT97wsQT6dm rYFddfh7Bcd2uDvIv8OK8YrWcxD0kkYz3BaBF2D1o91fFZck6DJ3goT+tlnlFuAaaSDqmI SyBZ9PSET+Lj95KTBTnxPRz1BZymfbGFh2KI6QfoVX8FiKeIy8gGGuHwahBva/pSxlwJ1L wosYvhq1Bb2WYggWSIDtHU5jFGbZgRnv0Nldu2TgcQI+aeJIyMdMEaS5uXwTOg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1642872957; a=rsa-sha256; cv=none; b=B7wmYiUnL6XEi7zIp0x8fQoVRpTnVMXoZo0o6eqpOu5qa5XYkzWm0Daq37mBRWY2N6z4mG 2QBXaxUjkpK521Z3rFE4nZgGc6Jt4P5jghBa5mxX9LyDThEsJAM2FeVa7iHuc/djK6kXFa SS7px0khgtNg9g+FcgFcXhnRFQ4TeVAILU/ZOzpsxJHA4kjlOWGMQbGP4XlT0aRP8IMTUP jpS1AhLOGz0eqH+9DlxQfGjbmhstK6uEBz/2nKADOi3p6smXIBa5Yzur1sbjohUVj1p5OJ PZPpssIw7VGBV5BnG2GN0f9zQBnpWWGNOTrPW9326HlrXAY6e8Z/VuflA9q84Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=3de96d664aaaf8e3fb1ca4fc4bd864d2cf734b24 commit 3de96d664aaaf8e3fb1ca4fc4bd864d2cf734b24 Author: Konstantin Belousov AuthorDate: 2022-01-16 20:18:21 +0000 Commit: Konstantin Belousov CommitDate: 2022-01-22 17:34:32 +0000 vm_pageout_scans: correct detection of active object For non-anonymous swap objects, there is always a reference from the owner to the object to keep it from recycling. Account for it when deciding should we query pmap for hardware active references for the page. As result, we avoid unneeded calls to pmap_ts_referenced(), which for non-mapped page means avoiding unneccessary lock and unlock of the pv list. Reviewed by: markj Discussed with: alc Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D33924 --- sys/vm/vm_pageout.c | 56 ++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 18 deletions(-) diff --git a/sys/vm/vm_pageout.c b/sys/vm/vm_pageout.c index 36d5f3275800..7d5c90c78f83 100644 --- a/sys/vm/vm_pageout.c +++ b/sys/vm/vm_pageout.c @@ -712,6 +712,38 @@ unlock_mp: return (error); } +/* + * Check if the object is active. Non-anonymous swap objects are + * always referenced by the owner, for them require ref_count > 1 in + * order to ignore the ownership ref. + * + * Perform an unsynchronized object ref count check. While + * the page lock ensures that the page is not reallocated to + * another object, in particular, one with unmanaged mappings + * that cannot support pmap_ts_referenced(), two races are, + * nonetheless, possible: + * 1) The count was transitioning to zero, but we saw a non- + * zero value. pmap_ts_referenced() will return zero + * because the page is not mapped. + * 2) The count was transitioning to one, but we saw zero. + * This race delays the detection of a new reference. At + * worst, we will deactivate and reactivate the page. + */ +static bool +vm_pageout_object_act(vm_object_t object) +{ + return (object->ref_count > + ((object->flags & (OBJ_SWAP | OBJ_ANON)) == OBJ_SWAP ? 1 : 0)); +} + +static int +vm_pageout_page_ts_referenced(vm_object_t object, vm_page_t m) +{ + if (!vm_pageout_object_act(object)) + return (0); + return (pmap_ts_referenced(m)); +} + /* * Attempt to launder the specified number of pages. * @@ -806,7 +838,7 @@ scan: if (vm_page_none_valid(m)) goto free_page; - refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; + refs = vm_pageout_page_ts_referenced(object, m); for (old = vm_page_astate_load(m);;) { /* @@ -826,7 +858,7 @@ scan: } if (act_delta == 0) { ; - } else if (object->ref_count != 0) { + } else if (vm_pageout_object_act(object)) { /* * Increase the activation count if the page was * referenced while in the laundry queue. This @@ -1263,20 +1295,8 @@ act_scan: * Test PGA_REFERENCED after calling pmap_ts_referenced() so * that a reference from a concurrently destroyed mapping is * observed here and now. - * - * Perform an unsynchronized object ref count check. While - * the page lock ensures that the page is not reallocated to - * another object, in particular, one with unmanaged mappings - * that cannot support pmap_ts_referenced(), two races are, - * nonetheless, possible: - * 1) The count was transitioning to zero, but we saw a non- - * zero value. pmap_ts_referenced() will return zero - * because the page is not mapped. - * 2) The count was transitioning to one, but we saw zero. - * This race delays the detection of a new reference. At - * worst, we will deactivate and reactivate the page. */ - refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; + refs = vm_pageout_page_ts_referenced(object, m); old = vm_page_astate_load(m); do { @@ -1526,7 +1546,7 @@ vm_pageout_scan_inactive(struct vm_domain *vmd, int page_shortage) if (vm_page_none_valid(m)) goto free_page; - refs = object->ref_count != 0 ? pmap_ts_referenced(m) : 0; + refs = vm_pageout_page_ts_referenced(object, m); for (old = vm_page_astate_load(m);;) { /* @@ -1546,7 +1566,7 @@ vm_pageout_scan_inactive(struct vm_domain *vmd, int page_shortage) } if (act_delta == 0) { ; - } else if (object->ref_count != 0) { + } else if (vm_pageout_object_act(object)) { /* * Increase the activation count if the * page was referenced while in the @@ -1584,7 +1604,7 @@ vm_pageout_scan_inactive(struct vm_domain *vmd, int page_shortage) * mappings allow write access, then the page may still be * modified until the last of those mappings are removed. */ - if (object->ref_count != 0) { + if (vm_pageout_object_act(object)) { vm_page_test_dirty(m); if (m->dirty == 0 && !vm_page_try_remove_all(m)) goto skip_page;