From nobody Sun Aug 04 20:33:56 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcWW83Qvgz5SyHd; Sun, 04 Aug 2024 20:33:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcWW82nDwz4bXC; Sun, 4 Aug 2024 20:33:56 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1722803636; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=j6Hftsur/sNk/hiSqsyvXKPZyDBYjGBfOJFKRTFAQSM=; b=D/RES/zjoC831gEH0t+e4rFXNFmuXnfQSL8Ln56cJxGKN2OQjYeAlXJpS8WDMrVR+gE0+7 8AJAUcpIfJvTSIMJ/KUJ4scIHBaizmBIcEdQfGzVw7jY/G3Q3soSjGPKuYs9wOn7+/+Qdc CY9RWeBdvngiO8+PVd5xLcJUZA/FMxTJWD8zSqdH6IryQrrMMvVWiK9CyaR8O36QQihdLQ /e/XallCMfWMdOSKW3CAlz+A4Gw6t0nght18vAPU+hFCMC/fNGdGbH3VQBsHJVUZ96yAmU 4jUcCZyYepujQ9JxTukUHcJ4MTdKKCPavQdOd3SBffPiCv7ki2om+ARKtu0RQA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1722803636; a=rsa-sha256; cv=none; b=EBECxYr3x0rRj3hMTI6/88cMd03wcG+bzebga4Oi0gq2TDT+Fh8mWhjQv0rvMnVv2vMhH/ vep2+BzEjvDW+PVHk4HYTOhVJQPhrI/jvi6QLU0KNKIJutEvLVkmbgjMlifcx0Vib4gbPa ErTVhC7vPvZ8DyeL4LhoGLxWHQq5PRf2+pGAAIB/s4tR9QrhtbImTxTvHGBe2Pkbal4EQK A/u05UYyoxdnRxpNaVSDUzIgOJkr1FpDsc3KEUW/gPHJ9c7lJsPv4x4krskmQjRW4seCI0 l9p5dkPuND0reDVWfYxJoYNWHBUU5QtSpaXLjhtXMSBnKE9gqsfBBuAhlztyTw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1722803636; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=j6Hftsur/sNk/hiSqsyvXKPZyDBYjGBfOJFKRTFAQSM=; b=athPr8VzWZk9/eWeaOcOOFlr2IDS4Uw8rTTqXjik9u/u79VS0fwIwSgzcRDXTzaiK7j1Jr D4P7CwpEs9CnlWh8odc41oJj5nMugNJY+JIT0RCT6wszSI8egFchtsvEnmBLt/ng19Xhbv G9E1NRsJyNO17jfi6IlQz85W0I0vu5R+G9JGVvjm2geBDD3Wr25Psv3Q6d7zHMfJxeK1JH R7dvW2ozf7hqNlD63TCd1fKLmxHm63GxOKTpPVtuBrbOqbx7uxR64WzUv82C0yphg8ELXA tVLgOhDlXB1TfwI59yxbzFckO/0OfkoQzi1icJ4es2u3rxXCCa2kH0mYB8UatA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WcWW82NdMz18lp; Sun, 4 Aug 2024 20:33:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 474KXu0Y093579; Sun, 4 Aug 2024 20:33:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 474KXusM093576; Sun, 4 Aug 2024 20:33:56 GMT (envelope-from git) Date: Sun, 4 Aug 2024 20:33:56 GMT Message-Id: <202408042033.474KXusM093576@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Doug Moore Subject: git: acb4cb33d358 - main - vm_pageout: simplify pageout_cluster List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dougm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: acb4cb33d35838e3e86412202cd63d9021b21ce2 Auto-Submitted: auto-generated The branch main has been updated by dougm: URL: https://cgit.FreeBSD.org/src/commit/?id=acb4cb33d35838e3e86412202cd63d9021b21ce2 commit acb4cb33d35838e3e86412202cd63d9021b21ce2 Author: Doug Moore AuthorDate: 2024-08-04 20:32:15 +0000 Commit: Doug Moore CommitDate: 2024-08-04 20:32:15 +0000 vm_pageout: simplify pageout_cluster Rewrite vm_pageout_cluster to eliminate redundant variables and duplicated code. Remove tests on pindex to check for object boundary conditions, since the page_next and page_prev functions return NULL at the object boundaries. Fix an alignment error that could happen if pindex is aligned, and the first of vm_pageout_page_count flushable pages, and the page at pindex-1 is also flushable. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D46217 --- sys/vm/vm_pageout.c | 115 +++++++++++++++++++++------------------------------- 1 file changed, 46 insertions(+), 69 deletions(-) diff --git a/sys/vm/vm_pageout.c b/sys/vm/vm_pageout.c index 742e0afbc690..0d054da34866 100644 --- a/sys/vm/vm_pageout.c +++ b/sys/vm/vm_pageout.c @@ -339,6 +339,25 @@ vm_pageout_defer(vm_page_t m, const uint8_t queue, const bool enqueued) return (false); } +/* + * We can cluster only if the page is not clean, busy, or held, and the page is + * in the laundry queue. + */ +static bool +vm_pageout_flushable(vm_page_t m) +{ + if (vm_page_tryxbusy(m) == 0) + return (false); + if (!vm_page_wired(m)) { + vm_page_test_dirty(m); + if (m->dirty != 0 && vm_page_in_laundry(m) && + vm_page_try_remove_write(m)) + return (true); + } + vm_page_xunbusy(m); + return (false); +} + /* * Scan for pages at adjacent offsets within the given page's object that are * eligible for laundering, form a cluster of these pages and the given page, @@ -348,26 +367,21 @@ static int vm_pageout_cluster(vm_page_t m) { vm_object_t object; - vm_page_t mc[2 * vm_pageout_page_count - 1], p, pb, ps; - vm_pindex_t pindex; - int ib, is, page_base, pageout_count; + vm_page_t mc[2 * vm_pageout_page_count - 1]; + int alignment, num_ends, page_base, pageout_count; object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); - pindex = m->pindex; vm_page_assert_xbusied(m); - pageout_count = 1; + alignment = m->pindex % vm_pageout_page_count; + num_ends = 0; page_base = nitems(mc) / 2; - mc[page_base] = pb = ps = m; - ib = 1; - is = 1; + pageout_count = 1; + mc[page_base] = m; /* - * We can cluster only if the page is not clean, busy, or held, and - * the page is in the laundry queue. - * * During heavy mmap/modification loads the pageout * daemon can really fragment the underlying file * due to flushing pages out of order and not trying to @@ -377,74 +391,37 @@ vm_pageout_cluster(vm_page_t m) * forward scan if room remains. */ more: - while (ib != 0 && pageout_count < vm_pageout_page_count) { - if (ib > pindex) { - ib = 0; - break; - } - if ((p = vm_page_prev(pb)) == NULL || - vm_page_tryxbusy(p) == 0) { - ib = 0; - break; - } - if (vm_page_wired(p)) { - ib = 0; - vm_page_xunbusy(p); - break; - } - vm_page_test_dirty(p); - if (p->dirty == 0) { - ib = 0; - vm_page_xunbusy(p); - break; - } - if (!vm_page_in_laundry(p) || !vm_page_try_remove_write(p)) { - vm_page_xunbusy(p); - ib = 0; - break; - } - mc[--page_base] = pb = p; - ++pageout_count; - ++ib; - + m = mc[page_base]; + while (pageout_count < vm_pageout_page_count) { /* - * We are at an alignment boundary. Stop here, and switch - * directions. Do not clear ib. + * If we are at an alignment boundary, and haven't reached the + * last flushable page forward, stop here, and switch + * directions. */ - if ((pindex - (ib - 1)) % vm_pageout_page_count == 0) - break; - } - while (pageout_count < vm_pageout_page_count && - pindex + is < object->size) { - if ((p = vm_page_next(ps)) == NULL || - vm_page_tryxbusy(p) == 0) + if (alignment == pageout_count - 1 && num_ends == 0) break; - if (vm_page_wired(p)) { - vm_page_xunbusy(p); - break; - } - vm_page_test_dirty(p); - if (p->dirty == 0) { - vm_page_xunbusy(p); + + m = vm_page_prev(m); + if (m == NULL || !vm_pageout_flushable(m)) { + num_ends++; break; } - if (!vm_page_in_laundry(p) || !vm_page_try_remove_write(p)) { - vm_page_xunbusy(p); + mc[--page_base] = m; + ++pageout_count; + } + m = mc[page_base + pageout_count - 1]; + while (num_ends != 2 && pageout_count < vm_pageout_page_count) { + m = vm_page_next(m); + if (m == NULL || !vm_pageout_flushable(m)) { + if (num_ends++ == 0) + /* Resume the reverse scan. */ + goto more; break; } - mc[page_base + pageout_count] = ps = p; + mc[page_base + pageout_count] = m; ++pageout_count; - ++is; } - /* - * If we exhausted our forward scan, continue with the reverse scan - * when possible, even past an alignment boundary. This catches - * boundary conditions. - */ - if (ib != 0 && pageout_count < vm_pageout_page_count) - goto more; - return (vm_pageout_flush(&mc[page_base], pageout_count, VM_PAGER_PUT_NOREUSE, 0, NULL, NULL)); }