From nobody Wed Jun 12 03:54:50 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VzWrp2qjRz5Ncdt; Wed, 12 Jun 2024 03:54:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VzWrp1rr4z468T; Wed, 12 Jun 2024 03:54:50 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1718164490; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tX3lQ2WVcoB51bfG1ngqHvWsxHtQqnYlnrB8i8V7mLw=; b=Fp14KfoHDf+P9hgI2ffMgwkcR8/3kjHneIE86eDq/Hj8c+f2UZLP1Bh8f9pcyW6SJHfUzR 8SAF0pgsZjaHfnJksJVKL9/WmD9rX4iA/sTHKiSM1dopBKIQy5X7w2hxM3hGynCjgZ1Ic9 UgwwGLJ+PuXWr0K2+BQT4a4oOlTzZdQc/Fdg4u1kYRIQBN+qfYQYABDyQsSRNY5Jv0CDwy qGNnS3W/mfSsDEiVaZdFPFqSQnDFitc7qnSFWtyjRH9ciMq4o1rSS/oleAfIgQJtiu4EoO R07D2QPNRX+n5e9N8JmRc0T+ElBr5JYMFPQeO3DIZjdscWa6SKTkbQdXZ/Aggg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1718164490; a=rsa-sha256; cv=none; b=XmDF38f7yPSSS5twWrnB18ONFlqY6gUp50mfaMpsDk4ElpyqW7xwAOa8ATvhOpuljB2AIv NEm8HHHylHwE1m0/+B0ZxztEL16/se5UIkWqDsVbnihLCDROHsTtLPiLbQ/JavGu1Pcz5M bmYd6wWvIggtIeqy8smTOpYre9zfpdggf7OXrpPt7rv50fwbl6lcWUB7O01AZ41Ybk9h8q YqxmdZw9cp3rqCA9tJrZ00J8Jpapkl4JXJWQlk2ddldETI/rVU5e+cIVSCL6oIkgvQsAn9 OjRugOpw+ucKca3LUCJoi4alHg2T20KG6WwMkqTWpxSMRTPdS2YzxEC6IvynEA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1718164490; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tX3lQ2WVcoB51bfG1ngqHvWsxHtQqnYlnrB8i8V7mLw=; b=ZUb/bUeudWfleyOXXFdQb53jfWwd14NoVB6gb8B+c8b82m2kNJkv8mMyvpiYkLQK+a8DCi FWI+hJNqodWAzUa7olCuyXcJFu8PUi+TaSr2fYPd/at5P0k/8S/nMC+kbl7ciwgi7RW15d L2RFLoCj0c0FSW+LQmMRhopL0wIxCwWi5O2zjHyjWds007gAfWJyoxdIm741H8+bbF8WgI Wacyq2fqso1z8gsXLBBMhY41Ld75AEc2aSGQ1tqNcDq4cZY3YmU9poBzrOtrH3GsmynYvT D30CsgM7jxWaP5tFdwGMwUssidsCX//2gfcqOOrRqmFgyiKX7WaSomJluOsEaA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VzWrp13DczdPL; Wed, 12 Jun 2024 03:54:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 45C3sor1047138; Wed, 12 Jun 2024 03:54:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 45C3soYJ047136; Wed, 12 Jun 2024 03:54:50 GMT (envelope-from git) Date: Wed, 12 Jun 2024 03:54:50 GMT Message-Id: <202406120354.45C3soYJ047136@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Doug Moore Subject: git: a880104a21bf - main - swap_pager: add new page range struct List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dougm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a880104a21bf41ebbb4ead26e6d4adda32bad76c Auto-Submitted: auto-generated The branch main has been updated by dougm: URL: https://cgit.FreeBSD.org/src/commit/?id=a880104a21bf41ebbb4ead26e6d4adda32bad76c commit a880104a21bf41ebbb4ead26e6d4adda32bad76c Author: Doug Moore AuthorDate: 2024-06-12 03:51:40 +0000 Commit: Doug Moore CommitDate: 2024-06-12 03:54:39 +0000 swap_pager: add new page range struct Define a page_range struct to pair up the two values passed to freerange functions. Have swp_pager_freeswapspace also take a page_range argument rather than a pair of arguments. In swp_pager_meta_free_all, drop a needless test and use a new helper function to do the cleanup for each swap block. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D45562 --- sys/vm/swap_pager.c | 99 ++++++++++++++++++++++++++++++----------------------- 1 file changed, 56 insertions(+), 43 deletions(-) diff --git a/sys/vm/swap_pager.c b/sys/vm/swap_pager.c index 15f227457bba..3bfda3eea169 100644 --- a/sys/vm/swap_pager.c +++ b/sys/vm/swap_pager.c @@ -142,6 +142,15 @@ struct swblk { daddr_t d[SWAP_META_PAGES]; }; +/* + * A page_range structure records the start address and length of a sequence of + * mapped page addresses. + */ +struct page_range { + daddr_t start; + daddr_t num; +}; + static MALLOC_DEFINE(M_VMPGDATA, "vm_pgdata", "swap pager private data"); static struct mtx sw_dev_mtx; static TAILQ_HEAD(, swdevt) swtailq = TAILQ_HEAD_INITIALIZER(swtailq); @@ -471,7 +480,7 @@ static int swapoff_one(struct swdevt *sp, struct ucred *cred, /* * Swap bitmap functions */ -static void swp_pager_freeswapspace(daddr_t blk, daddr_t npages); +static void swp_pager_freeswapspace(const struct page_range *range); static daddr_t swp_pager_getswapspace(int *npages); /* @@ -486,23 +495,21 @@ static void swp_pager_meta_free_all(vm_object_t); static daddr_t swp_pager_meta_lookup(vm_object_t, vm_pindex_t); static void -swp_pager_init_freerange(daddr_t *start, daddr_t *num) +swp_pager_init_freerange(struct page_range *range) { - - *start = SWAPBLK_NONE; - *num = 0; + range->start = SWAPBLK_NONE; + range->num = 0; } static void -swp_pager_update_freerange(daddr_t *start, daddr_t *num, daddr_t addr) +swp_pager_update_freerange(struct page_range *range, daddr_t addr) { - - if (*start + *num == addr) { - (*num)++; + if (range->start + range->num == addr) { + range->num++; } else { - swp_pager_freeswapspace(*start, *num); - *start = addr; - *num = 1; + swp_pager_freeswapspace(range); + range->start = addr; + range->num = 1; } } @@ -906,10 +913,13 @@ swp_pager_strategy(struct buf *bp) * This routine may not sleep. */ static void -swp_pager_freeswapspace(daddr_t blk, daddr_t npages) +swp_pager_freeswapspace(const struct page_range *range) { + daddr_t blk, npages; struct swdevt *sp; + blk = range->start; + npages = range->num; if (npages == 0) return; mtx_lock(&sw_dev_mtx); @@ -1004,11 +1014,12 @@ swap_pager_freespace_pgo(vm_object_t object, vm_pindex_t start, vm_size_t size) int swap_pager_reserve(vm_object_t object, vm_pindex_t start, vm_pindex_t size) { - daddr_t addr, blk, n_free, s_free; + struct page_range range; + daddr_t addr, blk; vm_pindex_t i, j; int n; - swp_pager_init_freerange(&s_free, &n_free); + swp_pager_init_freerange(&range); VM_OBJECT_WLOCK(object); for (i = 0; i < size; i += n) { n = MIN(size - i, INT_MAX); @@ -1022,11 +1033,10 @@ swap_pager_reserve(vm_object_t object, vm_pindex_t start, vm_pindex_t size) addr = swp_pager_meta_build(object, start + i + j, blk + j); if (addr != SWAPBLK_NONE) - swp_pager_update_freerange(&s_free, &n_free, - addr); + swp_pager_update_freerange(&range, addr); } } - swp_pager_freeswapspace(s_free, n_free); + swp_pager_freeswapspace(&range); VM_OBJECT_WUNLOCK(object); return (0); } @@ -1195,6 +1205,7 @@ swap_pager_haspage(vm_object_t object, vm_pindex_t pindex, int *before, static void swap_pager_unswapped(vm_page_t m) { + struct page_range range; struct swblk *sb; vm_object_t obj; @@ -1233,9 +1244,11 @@ swap_pager_unswapped(vm_page_t m) rounddown(m->pindex, SWAP_META_PAGES)); if (sb == NULL) return; - if (sb->d[m->pindex % SWAP_META_PAGES] == SWAPBLK_NONE) + range.start = sb->d[m->pindex % SWAP_META_PAGES]; + if (range.start == SWAPBLK_NONE) return; - swp_pager_freeswapspace(sb->d[m->pindex % SWAP_META_PAGES], 1); + range.num = 1; + swp_pager_freeswapspace(&range); sb->d[m->pindex % SWAP_META_PAGES] = SWAPBLK_NONE; swp_pager_free_empty_swblk(m->object, sb); } @@ -1480,8 +1493,9 @@ static void swap_pager_putpages(vm_object_t object, vm_page_t *ma, int count, int flags, int *rtvals) { + struct page_range range; struct buf *bp; - daddr_t addr, blk, n_free, s_free; + daddr_t addr, blk; vm_page_t mreq; int i, j, n; bool async; @@ -1492,7 +1506,7 @@ swap_pager_putpages(vm_object_t object, vm_page_t *ma, int count, VM_OBJECT_WUNLOCK(object); async = curproc == pageproc && (flags & VM_PAGER_PUT_SYNC) == 0; - swp_pager_init_freerange(&s_free, &n_free); + swp_pager_init_freerange(&range); /* * Assign swap blocks and issue I/O. We reallocate swap on the fly. @@ -1530,8 +1544,7 @@ swap_pager_putpages(vm_object_t object, vm_page_t *ma, int count, addr = swp_pager_meta_build(mreq->object, mreq->pindex, blk + j); if (addr != SWAPBLK_NONE) - swp_pager_update_freerange(&s_free, &n_free, - addr); + swp_pager_update_freerange(&range, addr); MPASS(mreq->dirty == VM_PAGE_BITS_ALL); mreq->oflags |= VPO_SWAPINPROG; } @@ -1603,7 +1616,7 @@ swap_pager_putpages(vm_object_t object, vm_page_t *ma, int count, */ swp_pager_async_iodone(bp); } - swp_pager_freeswapspace(s_free, n_free); + swp_pager_freeswapspace(&range); VM_OBJECT_WLOCK(object); } @@ -2131,9 +2144,9 @@ static void swp_pager_meta_transfer(vm_object_t srcobject, vm_object_t dstobject, vm_pindex_t pindex, vm_pindex_t count, vm_size_t *moved) { + struct page_range range; struct swblk *sb; vm_page_t m; - daddr_t n_free, s_free; vm_pindex_t offset, last; vm_size_t mc; int i, limit, start; @@ -2146,7 +2159,7 @@ swp_pager_meta_transfer(vm_object_t srcobject, vm_object_t dstobject, if (count == 0 || pctrie_is_empty(&srcobject->un_pager.swp.swp_blks)) goto out; - swp_pager_init_freerange(&s_free, &n_free); + swp_pager_init_freerange(&range); offset = pindex; last = pindex + count; for (;;) { @@ -2163,8 +2176,7 @@ swp_pager_meta_transfer(vm_object_t srcobject, vm_object_t dstobject, if (dstobject == NULL || !swp_pager_xfer_source(srcobject, dstobject, sb->p + i - offset, sb->d[i])) { - swp_pager_update_freerange(&s_free, &n_free, - sb->d[i]); + swp_pager_update_freerange(&range, sb->d[i]); } if (moved != NULL) { if (m != NULL && m->pindex != pindex + i - 1) @@ -2184,7 +2196,7 @@ swp_pager_meta_transfer(vm_object_t srcobject, vm_object_t dstobject, uma_zfree(swblk_zone, sb); } } - swp_pager_freeswapspace(s_free, n_free); + swp_pager_freeswapspace(&range); out: if (moved != NULL) *moved = mc; @@ -2207,6 +2219,16 @@ swp_pager_meta_free(vm_object_t object, vm_pindex_t pindex, vm_pindex_t count, swp_pager_meta_transfer(object, NULL, pindex, count, freed); } +static void +swp_pager_meta_free_block(struct swblk *sb, struct page_range *range) +{ + for (int i = 0; i < SWAP_META_PAGES; i++) { + if (sb->d[i] != SWAPBLK_NONE) + swp_pager_update_freerange(range, sb->d[i]); + } + uma_zfree(swblk_zone, sb); +} + /* * SWP_PAGER_META_FREE_ALL() - destroy all swap metadata associated with object * @@ -2217,28 +2239,19 @@ static void swp_pager_meta_free_all(vm_object_t object) { struct swblk *sb; - daddr_t n_free, s_free; + struct page_range range; vm_pindex_t pindex; - int i; VM_OBJECT_ASSERT_WLOCKED(object); - if (pctrie_is_empty(&object->un_pager.swp.swp_blks)) - return; - - swp_pager_init_freerange(&s_free, &n_free); + swp_pager_init_freerange(&range); for (pindex = 0; (sb = SWAP_PCTRIE_LOOKUP_GE( &object->un_pager.swp.swp_blks, pindex)) != NULL;) { pindex = sb->p + SWAP_META_PAGES; - for (i = 0; i < SWAP_META_PAGES; i++) { - if (sb->d[i] == SWAPBLK_NONE) - continue; - swp_pager_update_freerange(&s_free, &n_free, sb->d[i]); - } SWAP_PCTRIE_REMOVE(&object->un_pager.swp.swp_blks, sb->p); - uma_zfree(swblk_zone, sb); + swp_pager_meta_free_block(sb, &range); } - swp_pager_freeswapspace(s_free, n_free); + swp_pager_freeswapspace(&range); } /*