From nobody Wed Jul 31 15:51:27 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYxR43rxGz5S798; Wed, 31 Jul 2024 15:51:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYxR41HNYz41Q2; Wed, 31 Jul 2024 15:51:28 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1722441088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QXsPn2WYR/082xT286qWN6Hb/y6Pnsl9sQH67Q3b4jg=; b=CtUWHHltNCxQHS3DpuEqWPfKQ750D2fxEWU+jLVdOKCbABTPHg0DBHYxO+i1oWrQ5wgC5G NUZOSZWPKrzk2JH41dfYvwzlbJuVgHNlvnkj75mkFic98XuDw4MlNvvb1Ufs8kkA7RbCCz ck+FbOzeFEXztEUMgQDnu5ookKin6umim3yO+vSDf0W1/ExQTgFtrtUfGk5SacOqTZ693V fV6uB8O5nfQ4PztpSqIArqxwxpUrXAqRLq8B+uq+t3KP0i+pADEYJM8oVb3/qSPLOJqVuF /Zm2/r5TMOzb9tfmJPrEuJzsafc9Y9MGdLRJEWNNTisHJvsfaudW2INdnpX+VQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1722441088; a=rsa-sha256; cv=none; b=BWndz+8HpZx9JvwCoNtlrc/rgyPxIbGEC8Nw0M9TPMd9v4k0PGVdvqr+0h9AtScjg1bhGJ fcHUdODxE8cJ7dnvghn+of8ZFjocd7IWAQ3cmSqcpyVqYM72PhdtInVlFJ4tuO8WsCZl3l tM4zUGJv0rXoIE/jHsS45wu2j4xQyj/RnypkOG+4gKqKfo4jkjFkFFrksOcs63YiDg2zmU 4jg4//h30s/x8nIFShe3EerbAv+ONETjfZjIBmq/QusVS3Le8zCyuhWFeyJcurZL8XjMER Z1IIrgriTPh+uq6d41eeOBJhSzGOJ1kbI6yMf1pM9yierPgpUNKSjVHzSzG+7g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1722441088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QXsPn2WYR/082xT286qWN6Hb/y6Pnsl9sQH67Q3b4jg=; b=lfPRjloT4Dj89hMUVa1j+c6PJ8iG1w1g3dsEkudR5mG9MzmyT7SEoP0NhZaatAZbqqBYd9 mhISkHXbgyJOkBlhGcbSGTCDOBHkzVJ+9OEus9moUjewj/m/8sKJKl11nI9qkQBfFLHNGE SrW8R+6zrjjIkJapESJ4bO/s9urupufyFjfBD7MrcgamaStkAqHh0QJma4qKiPWjwVg2j0 djiDykcxYNnmQNDEEdtilihCvAIjGX6B5IaHYwafwX8K2Z6FTcpi2EgZrG1XhxeTUtJWbh 31k4AqfasdnYuXEkbJowk9ecEVO24LMcHqyctKVBubx+IZTsRhcZuwtoki21xQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WYxR40g2Nz1689; Wed, 31 Jul 2024 15:51:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46VFpRpv014442; Wed, 31 Jul 2024 15:51:27 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46VFpRcr014439; Wed, 31 Jul 2024 15:51:27 GMT (envelope-from git) Date: Wed, 31 Jul 2024 15:51:27 GMT Message-Id: <202407311551.46VFpRcr014439@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mitchell Horne Subject: git: 46bc4963e2de - main - riscv: implement pmap_demote_l1() List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mhorne X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 46bc4963e2de8d776c55015f61af129f00ad5b46 Auto-Submitted: auto-generated The branch main has been updated by mhorne: URL: https://cgit.FreeBSD.org/src/commit/?id=46bc4963e2de8d776c55015f61af129f00ad5b46 commit 46bc4963e2de8d776c55015f61af129f00ad5b46 Author: Mitchell Horne AuthorDate: 2024-07-31 14:06:10 +0000 Commit: Mitchell Horne CommitDate: 2024-07-31 15:28:13 +0000 riscv: implement pmap_demote_l1() For use in pmap_change_attr_locked(), where we might need to demote L1 pages in the DMAP. Reviewed by: markj MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D45628 --- sys/riscv/riscv/pmap.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/sys/riscv/riscv/pmap.c b/sys/riscv/riscv/pmap.c index 34922b0f511e..5eb5c02b7461 100644 --- a/sys/riscv/riscv/pmap.c +++ b/sys/riscv/riscv/pmap.c @@ -279,6 +279,13 @@ SYSCTL_ULONG(_vm_pmap_l2, OID_AUTO, promotions, CTLFLAG_RD, &pmap_l2_promotions, 0, "2MB page promotions"); +static SYSCTL_NODE(_vm_pmap, OID_AUTO, l1, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, + "L1 (1GB) page mapping counters"); + +static COUNTER_U64_DEFINE_EARLY(pmap_l1_demotions); +SYSCTL_COUNTER_U64(_vm_pmap_l1, OID_AUTO, demotions, CTLFLAG_RD, + &pmap_l1_demotions, "L1 (1GB) page demotions"); + /* * Data for the pv entry allocation mechanism */ @@ -303,6 +310,7 @@ static vm_page_t reclaim_pv_chunk(pmap_t locked_pmap, struct rwlock **lockp); static void pmap_pvh_free(struct md_page *pvh, pmap_t pmap, vm_offset_t va); static pv_entry_t pmap_pvh_remove(struct md_page *pvh, pmap_t pmap, vm_offset_t va); +static bool pmap_demote_l1(pmap_t pmap, pd_entry_t *l1, vm_offset_t va); static bool pmap_demote_l2(pmap_t pmap, pd_entry_t *l2, vm_offset_t va); static bool pmap_demote_l2_locked(pmap_t pmap, pd_entry_t *l2, vm_offset_t va, struct rwlock **lockp); @@ -2795,6 +2803,67 @@ done: return (rv); } +/* + * Demote the specified L1 page to separate L2 pages. + * Currently only used for DMAP entries. + */ +static bool +pmap_demote_l1(pmap_t pmap, pd_entry_t *l1, vm_offset_t va) +{ + vm_page_t m; + pt_entry_t *l2, oldl1, newl2; + pd_entry_t newl1; + vm_paddr_t l2phys; + + PMAP_LOCK_ASSERT(pmap, MA_OWNED); + + oldl1 = pmap_load(l1); + KASSERT((oldl1 & PTE_RWX) != 0, + ("pmap_demote_l1: oldl1 is not a leaf PTE")); + KASSERT((oldl1 & PTE_A) != 0, + ("pmap_demote_l1: oldl1 is missing PTE_A")); + KASSERT((oldl1 & (PTE_D | PTE_W)) != PTE_W, + ("pmap_demote_l1: not dirty!")); + KASSERT((oldl1 & PTE_SW_MANAGED) == 0, + ("pmap_demote_l1: L1 table shouldn't be managed")); + KASSERT(VIRT_IN_DMAP(va), + ("pmap_demote_l1: is unsupported for non-DMAP va=%#lx", va)); + + /* Demoting L1 means we need to allocate a new page-table page. */ + m = vm_page_alloc_noobj(VM_ALLOC_INTERRUPT | VM_ALLOC_WIRED); + if (m == NULL) { + CTR2(KTR_PMAP, "pmap_demote_l1: failure for va %#lx in pmap %p", + va, pmap); + return (false); + } + + l2phys = VM_PAGE_TO_PHYS(m); + l2 = (pt_entry_t *)PHYS_TO_DMAP(l2phys); + + /* + * Create new entries, relying on the fact that only the low bits + * (index) of the physical address are changing. + */ + newl2 = oldl1; + for (int i = 0; i < Ln_ENTRIES; i++) + pmap_store(&l2[i], newl2 | (i << PTE_PPN1_S)); + + /* + * And update the L1 entry. + * + * NB: flushing the TLB is the responsibility of the caller. Cached + * translations are still "correct" for demoted mappings until some + * subset of the demoted range is modified. + */ + newl1 = ((l2phys / PAGE_SIZE) << PTE_PPN0_S) | PTE_V; + pmap_store(l1, newl1); + + counter_u64_add(pmap_l1_demotions, 1); + CTR2(KTR_PMAP, "pmap_demote_l1: success for va %#lx in pmap %p", + va, pmap); + return (true); +} + static bool pmap_demote_l2(pmap_t pmap, pd_entry_t *l2, vm_offset_t va) {