From nobody Wed Nov 20 18:04:09 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xtq4T6LhBz5dYh3; Wed, 20 Nov 2024 18:04:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xtq4T4Qr2z4WZn; Wed, 20 Nov 2024 18:04:09 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732125849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yc151jo85SJzrTDcXyKKonBPnpKPtZ6MUeCgDKbW8a0=; b=IX4tYUpjQP1j2nG/hSV8tTg6H8niwyC5zvDBvQXyUUWgYeVXBs8sfsD1yirf/daP9Y+LQ3 L/tipK7khKG9yLdj5BB0wiCDoIODIdHjwjZ4doEzTtgkXfZenSn3HIelA7utdYpJTxzIFR PkyikvFXwn2TsXvHlp1n834bLXaIVLiAIo3p0DqH2hZbqrGCSm8X7g1JE2GyhyHQxSX4e6 1Fnd+t6bf/jU5PpbfarkhEiJke53Qns9g2Em4ip/wgbP7ZppxLAOHmzutcdhwQi1FAstci g37F4TXMphbmtlG+/VCzLHKbz2bRaUwpuGneGqAf5s3FZB0YejW2p8eCxJcVvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732125849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yc151jo85SJzrTDcXyKKonBPnpKPtZ6MUeCgDKbW8a0=; b=J69KE0VQlyw/VQ6oBxSw+EhToUKH76cqYp3cF5fJb/kY/5+d1uITlx71tVGJdrl0N9TcrM kTeNs56fSKQFTJQPDBCL43IBLveB1Y1kek2XLTYTHSY4QviPPbi6O0IlyUtEts+/Tgm3w1 DfBj0wzCn71++H+ls1e/kjSyO2I1GTMxUbIM4H+3YVf7PxjDLmg1tdTVqzgy6Ej7fxAD2d UpAo4doKI4PBwk49OYBnkj8Ki3yUkTtsgRbAvQLa1qpV/h4GSD0NdzQQNQjAb/+VNzk91a AF5e/aTaqTSRZCtjUng7Zh/Rv32HHdIo+viWuDOVJVI1cWc8lj1yiEJVH24Emg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732125849; a=rsa-sha256; cv=none; b=YWLpgGDmTr0R5NdKqUaYskmDbXyJK+5/EXrNaoGIVXzY9TG8PGTgkKWvSm0g9k8vV6WYld 4peYvad31CqmrKqKXPKvliTDcplGbn6DBnFSAE78LDASfHXCYH594FbKVsnQ0DFkYHpE8k 7Z2PA5UGGxjZ/aiW/T744LnhKLwYiF3tKM+k6VMwiBWUnrMrtVYfdWJQl9ujtrUGt0Gzfa o8mjBD7aSbBAMomzxxu1arRy0VfMis6ON5xMjVlFZG8ANGu/cZGYUQsDmfHcXIcpG8E6Me THzHorRbWEKsldKcFpkdgzLlSfsMjzI9eBbvoXzCkxId26SvDZGm9/TFlH4ZZA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Xtq4T42TWz14tF; Wed, 20 Nov 2024 18:04:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 4AKI49nu019189; Wed, 20 Nov 2024 18:04:09 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 4AKI497E019186; Wed, 20 Nov 2024 18:04:09 GMT (envelope-from git) Date: Wed, 20 Nov 2024 18:04:09 GMT Message-Id: <202411201804.4AKI497E019186@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Doug Moore Subject: git: 18a8f4e586b1 - main - vm_page: correct page iterator patch List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dougm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 18a8f4e586b1a5f4e01da7e5f76099a69a3dd1f8 Auto-Submitted: auto-generated The branch main has been updated by dougm: URL: https://cgit.FreeBSD.org/src/commit/?id=18a8f4e586b1a5f4e01da7e5f76099a69a3dd1f8 commit 18a8f4e586b1a5f4e01da7e5f76099a69a3dd1f8 Author: Doug Moore AuthorDate: 2024-11-20 18:00:57 +0000 Commit: Doug Moore CommitDate: 2024-11-20 18:00:57 +0000 vm_page: correct page iterator patch The previous change committed a preliminary version of the change to use iterators to free page sequences. This updates to what was intended to be the final version. Reviewed by: markj (previous version) Tested by: pho Differential Revision: https://reviews.freebsd.org/D46724 --- sys/vm/vm_page.c | 118 +++++++++++++++++++++++++++++-------------------------- 1 file changed, 63 insertions(+), 55 deletions(-) diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c index 7d093579e35d..296d803ca0f0 100644 --- a/sys/vm/vm_page.c +++ b/sys/vm/vm_page.c @@ -170,7 +170,7 @@ static bool _vm_page_busy_sleep(vm_object_t obj, vm_page_t m, vm_pindex_t pindex, const char *wmesg, int allocflags, bool locked); static void vm_page_clear_dirty_mask(vm_page_t m, vm_page_bits_t pagebits); static void vm_page_enqueue(vm_page_t m, uint8_t queue); -static bool vm_page_free_prep(vm_page_t m, bool do_remove); +static bool vm_page_free_prep(vm_page_t m); static void vm_page_free_toq(vm_page_t m); static void vm_page_free_toq_impl(vm_page_t m, bool do_remove); static void vm_page_init(void *dummy); @@ -1387,22 +1387,6 @@ vm_page_free(vm_page_t m) vm_page_free_toq(m); } -/* - * vm_page_iter_free: - * - * Free the current page, as identified by iterator. - */ -void -vm_page_iter_free(struct pctrie_iter *pages) -{ - vm_page_t m; - - m = vm_radix_iter_page(pages); - vm_radix_iter_remove(pages); - m->flags &= ~PG_ZERO; - vm_page_free_toq_impl(m, false); -} - /* * vm_page_free_zero: * @@ -1699,6 +1683,52 @@ vm_page_remove_radixdone(vm_page_t m) vdrop(object->handle); } +/* + * vm_page_free_object_prep: + * + * Disassociates the given page from its VM object. + * + * The object must be locked, and the page must be xbusy. + */ +static void +vm_page_free_object_prep(vm_page_t m) +{ + KASSERT(((m->oflags & VPO_UNMANAGED) != 0) == + ((m->object->flags & OBJ_UNMANAGED) != 0), + ("%s: managed flag mismatch for page %p", + __func__, m)); + vm_page_assert_xbusied(m); + + /* + * The object reference can be released without an atomic + * operation. + */ + KASSERT((m->flags & PG_FICTITIOUS) != 0 || + m->ref_count == VPRC_OBJREF, + ("%s: page %p has unexpected ref_count %u", + __func__, m, m->ref_count)); + vm_page_remove_radixdone(m); + m->ref_count -= VPRC_OBJREF; +} + +/* + * vm_page_iter_free: + * + * Free the current page, as identified by iterator. + */ +void +vm_page_iter_free(struct pctrie_iter *pages) +{ + vm_page_t m; + + m = vm_radix_iter_page(pages); + vm_radix_iter_remove(pages); + vm_page_free_object_prep(m); + vm_page_xunbusy(m); + m->flags &= ~PG_ZERO; + vm_page_free_toq(m); +} + /* * vm_page_remove: * @@ -3180,7 +3210,7 @@ vm_page_reclaim_run(int req_class, int domain, u_long npages, vm_page_t m_run, vm_page_dequeue(m); if (vm_page_replace_hold(m_new, object, m->pindex, m) && - vm_page_free_prep(m, true)) + vm_page_free_prep(m)) SLIST_INSERT_HEAD(&free, m, plinks.s.ss); @@ -3192,7 +3222,7 @@ vm_page_reclaim_run(int req_class, int domain, u_long npages, vm_page_t m_run, } else { m->flags &= ~PG_ZERO; vm_page_dequeue(m); - if (vm_page_free_prep(m, true)) + if (vm_page_free_prep(m)) SLIST_INSERT_HEAD(&free, m, plinks.s.ss); KASSERT(m->dirty == 0, @@ -4131,7 +4161,7 @@ vm_page_enqueue(vm_page_t m, uint8_t queue) * page must be unmapped. */ static bool -vm_page_free_prep(vm_page_t m, bool do_remove) +vm_page_free_prep(vm_page_t m) { /* @@ -4164,24 +4194,8 @@ vm_page_free_prep(vm_page_t m, bool do_remove) VM_CNT_INC(v_tfree); if (m->object != NULL) { - KASSERT(((m->oflags & VPO_UNMANAGED) != 0) == - ((m->object->flags & OBJ_UNMANAGED) != 0), - ("vm_page_free_prep: managed flag mismatch for page %p", - m)); - vm_page_assert_xbusied(m); - - /* - * The object reference can be released without an atomic - * operation. - */ - KASSERT((m->flags & PG_FICTITIOUS) != 0 || - m->ref_count == VPRC_OBJREF, - ("vm_page_free_prep: page %p has unexpected ref_count %u", - m, m->ref_count)); - if (do_remove) - vm_page_radix_remove(m); - vm_page_remove_radixdone(m); - m->ref_count -= VPRC_OBJREF; + vm_page_radix_remove(m); + vm_page_free_object_prep(m); } else vm_page_assert_unbusied(m); @@ -4232,13 +4246,22 @@ vm_page_free_prep(vm_page_t m, bool do_remove) return (true); } +/* + * vm_page_free_toq: + * + * Returns the given page to the free list, disassociating it + * from any VM object. + * + * The object must be locked. The page must be exclusively busied if it + * belongs to an object. + */ static void -vm_page_free_toq_impl(vm_page_t m, bool do_remove) +vm_page_free_toq(vm_page_t m) { struct vm_domain *vmd; uma_zone_t zone; - if (!vm_page_free_prep(m, do_remove)) + if (!vm_page_free_prep(m)) return; vmd = vm_pagequeue_domain(m); @@ -4253,21 +4276,6 @@ vm_page_free_toq_impl(vm_page_t m, bool do_remove) vm_domain_freecnt_inc(vmd, 1); } -/* - * vm_page_free_toq: - * - * Returns the given page to the free list, disassociating it - * from any VM object. - * - * The object must be locked. The page must be exclusively busied if it - * belongs to an object. - */ -static void -vm_page_free_toq(vm_page_t m) -{ - vm_page_free_toq_impl(m, true); -} - /* * vm_page_free_pages_toq: *