From nobody Thu Sep 19 20:24:19 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X8n6r1BYhz5XLMr; Thu, 19 Sep 2024 20:24:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X8n6r0KrJz4cZT; Thu, 19 Sep 2024 20:24:20 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1726777460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=2pEs6NUkdsZFhgV44dHqOFgltX6loHy0Cc+t15t2sSM=; b=deJkIKJSd9IKav3UZvdEP3EWN65+tg6lIJxYsg10gQvrEAkH49SsJn7Gkussua2ke3DRI9 jg1w/iNRg6uW2Q7zwnFVx9RgF5NeTNohu8dCq2nPIbM9+KrW1uOSKHv4JtQiNLxu0NQAOM A4LBB1cTtItFAT4Bl35EXFS50skK085LHFfO8pV/S7kQSdn+4Izy4hcwO/aY14GMEyYTMx R6Ht0OZtDnpotv/uqmv57c1838uFtTUSltsyHu/U0UCqPIrBIHxX6mzByq2yxNK0XE6PdE DRdmq5SJPn0ebxRu93vGjMCDd4MVoRb9Ssp/BxsokZTlyKxtChvzBs+F43QlSg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1726777460; a=rsa-sha256; cv=none; b=U6Zr4CcpDwDXZc1NbQSKrFjJ4kWz83DGQCtkQrcCKY08HU9p8OGrU+CAZiJ8wyxwr38MyP jmGn0frNPtEmkDQES65+0lijmnPjyAhyGn7BdzhtuS3bOAPdortaY9q9rJWTt8kuSejBry KGmd6W5C9rIgPKwoCmHoFCr3QM/L7hnffhG38Wkdd4U4FItRdHailX0zhT7KDoCb4i0yfu f/eHHHfPWETWeggv1p1NiUQRbrSPrp8vQ6ScZE4Qc36cHswMq4XW3/jpQhfaqu++T6y3I3 0t+zzXaMORpVkOj0yCKOdJfVSPOPVikdOaoqEKbqDVPKSU1Qx0z7VwMRBN9Jnw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1726777460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=2pEs6NUkdsZFhgV44dHqOFgltX6loHy0Cc+t15t2sSM=; b=P/jNPieVApWNdIgNEAWVBvLdRH9CbRynukDe+BWj1q2Xx9EuitXkIypM+FWx37kxfPsjx0 WkeufMPSTDin8vwbztJnh98Y5m/XsWnMaUWe3++/SCLq46o4W+fX55RaZcVyvT6Thm2IGH Zrzu2Z0wRl3qyFOQ4raApRSm5uFCWfqJKVFuMZBg+83rH53Ls4wQtbUa9qE9/wE5ZhQKJV aYsQlCmULQ+u4XUTfXgJAxFFyHm3hmE+d6sq2BVhvGzkznPJYeEoG+w5ExagaC477oHnIk NCpHsIfctj1hwiCkbvrdfkz+GHnjf4mRZajFKy3pThBSuuRkIVM/EjV58A1xWw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4X8n6q72zPzL8R; Thu, 19 Sep 2024 20:24:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 48JKOJCu052570; Thu, 19 Sep 2024 20:24:19 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 48JKOJ1U052567; Thu, 19 Sep 2024 20:24:19 GMT (envelope-from git) Date: Thu, 19 Sep 2024 20:24:19 GMT Message-Id: <202409192024.48JKOJ1U052567@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Doug Moore Subject: git: c3d743a67a7c - main - vm_object: use reclaim callback to free pages List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dougm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: c3d743a67a7c238645efe2979c40e423ae32f2ed Auto-Submitted: auto-generated The branch main has been updated by dougm: URL: https://cgit.FreeBSD.org/src/commit/?id=c3d743a67a7c238645efe2979c40e423ae32f2ed commit c3d743a67a7c238645efe2979c40e423ae32f2ed Author: Doug Moore AuthorDate: 2024-09-19 20:23:19 +0000 Commit: Doug Moore CommitDate: 2024-09-19 20:23:19 +0000 vm_object: use reclaim callback to free pages Instead of iterating over object->memq to free pages, use a callback in VM_RADIX_PCTRIE_RECLAIM_CALLBACK to do it. Reviewed by: rlibby Differential Revision: https://reviews.freebsd.org/D45588 --- sys/vm/vm_object.c | 62 ++++++++++++++++++++++++++++-------------------------- sys/vm/vm_radix.h | 10 +++++---- 2 files changed, 38 insertions(+), 34 deletions(-) diff --git a/sys/vm/vm_object.c b/sys/vm/vm_object.c index 7b3950b6f3bc..86f9271341df 100644 --- a/sys/vm/vm_object.c +++ b/sys/vm/vm_object.c @@ -876,6 +876,27 @@ vm_object_backing_collapse_wait(vm_object_t object) return (backing_object); } +/* + * vm_object_terminate_single_page removes a pageable page from the object, + * and removes it from the paging queues and frees it, if it is not wired. + * It is invoked via callback from vm_object_terminate_pages. + */ +static void +vm_object_terminate_single_page(vm_page_t p, void *objectv) +{ + vm_object_t object __diagused = objectv; + + vm_page_assert_unbusied(p); + KASSERT(p->object == object && + (p->ref_count & VPRC_OBJREF) != 0, + ("%s: page %p is inconsistent", __func__, p)); + p->object = NULL; + if (vm_page_drop(p, VPRC_OBJREF) == VPRC_OBJREF) { + VM_CNT_INC(v_pfree); + vm_page_free(p); + } +} + /* * vm_object_terminate_pages removes any remaining pageable pages * from the object and resets the object to an empty state. @@ -883,41 +904,22 @@ vm_object_backing_collapse_wait(vm_object_t object) static void vm_object_terminate_pages(vm_object_t object) { - vm_page_t p, p_next; - VM_OBJECT_ASSERT_WLOCKED(object); - /* - * Free any remaining pageable pages. This also removes them from the - * paging queues. However, don't free wired pages, just remove them - * from the object. Rather than incrementally removing each page from - * the object, the page and object are reset to any empty state. - */ - TAILQ_FOREACH_SAFE(p, &object->memq, listq, p_next) { - vm_page_assert_unbusied(p); - KASSERT(p->object == object && - (p->ref_count & VPRC_OBJREF) != 0, - ("vm_object_terminate_pages: page %p is inconsistent", p)); - - p->object = NULL; - if (vm_page_drop(p, VPRC_OBJREF) == VPRC_OBJREF) { - VM_CNT_INC(v_pfree); - vm_page_free(p); - } - } - /* * If the object contained any pages, then reset it to an empty state. - * None of the object's fields, including "resident_page_count", were - * modified by the preceding loop. + * Rather than incrementally removing each page from the object, the + * page and object are reset to any empty state. */ - if (object->resident_page_count != 0) { - vm_radix_reclaim_allnodes(&object->rtree); - TAILQ_INIT(&object->memq); - object->resident_page_count = 0; - if (object->type == OBJT_VNODE) - vdrop(object->handle); - } + if (object->resident_page_count == 0) + return; + + vm_radix_reclaim_callback(&object->rtree, + vm_object_terminate_single_page, object); + TAILQ_INIT(&object->memq); + object->resident_page_count = 0; + if (object->type == OBJT_VNODE) + vdrop(object->handle); } /* diff --git a/sys/vm/vm_radix.h b/sys/vm/vm_radix.h index f2e297b10641..d35c9a171d4b 100644 --- a/sys/vm/vm_radix.h +++ b/sys/vm/vm_radix.h @@ -187,14 +187,16 @@ vm_radix_remove(struct vm_radix *rtree, vm_pindex_t index) { return (VM_RADIX_PCTRIE_REMOVE_LOOKUP(&rtree->rt_trie, index)); } - + /* - * Reclaim all the interior nodes from the radix tree. + * Reclaim all the interior nodes of the trie, and invoke the callback + * on all the pages, in order. */ static __inline void -vm_radix_reclaim_allnodes(struct vm_radix *rtree) +vm_radix_reclaim_callback(struct vm_radix *rtree, + void (*page_cb)(vm_page_t, void *), void *arg) { - VM_RADIX_PCTRIE_RECLAIM(&rtree->rt_trie); + VM_RADIX_PCTRIE_RECLAIM_CALLBACK(&rtree->rt_trie, page_cb, arg); } /*