From nobody Sat Jun 24 19:08:30 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QpNtR1kX7z4hbQl; Sat, 24 Jun 2023 19:08:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QpNtR1HdPz4KxL; Sat, 24 Jun 2023 19:08:31 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687633711; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3qJMnZhOnVgNegz17W+266QRUBzDjp1+EtOQZ7l0RL8=; b=ukBe00RouAearNKKt9YbEpXzhHdanULbmkk24RRPQVxEXUBATJlss9MseHoHRs7WVOHZne wZhwkGI/X+6LXMAQ2rPlwtcvoq7vZnkM7U3pAxMxrSnM7OTxvoGWmmXauAmk8s8o2h2EZR S4I6Ia9MvGsPXG/1cV1iXNAwy52/JikVpFzXY4+1dGmkO0KrMyTfHK9BnOKP7K80tmPn0r IRZxHNMGm+jdcziP6/Q2oFKUHIBVXokDwHyG8xf/7nh1uoc23cqTysl9iXkGRaQId8Ywrd oflMFNopEyAPxsdA7y8qgowcm8GLFom35/fsHKbyNWkUadTvdwJhaOh2nSNUSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687633711; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3qJMnZhOnVgNegz17W+266QRUBzDjp1+EtOQZ7l0RL8=; b=waHSt6KMKF55f6ZpF6Jjv+N6L7fFzEeZ17R+sg/VXToodtLmQNoQ8Mu0CsETCPIEXbOAmF cQpGTT3UroG6QyDNv2cRFRiLyetlWNcK+5h/H0IUXqb8VXBJc1fOxirWZGla6bADS4WlRR JIPDAES6R2GpiOHhjlLXJFGfCrAxMZNXrWRQwp3rx1mIhtZvC9zZWt/A2rLx/xgxyBPFXD T4LMogzrdc0UsdkPxWtuYvjaVFIJB0OKI/2pxDzm/DJWyg8LNo6XrFfwnCyY/ji3teIZ/E 6LtfA/rnh25zSIwc7hgJgxyLXVQdHwx4RiRcwWWdP2yzw4d8sd0Iw5qnpFtBhQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1687633711; a=rsa-sha256; cv=none; b=CkthCmZGMeCpAgj1tEHC3VDDl2bFecJNT9QOtasjvphe8OhogGybYF9edL5I6zjWmjlgUw bvst7xGzhphA4OMW4AtPijwEp6N7FKRY0X0bYQ+jTsYI6PDzR+sp93JzN5i2RkNoCrLDAr jnlCqiZwye80Zucfcd1K8E1+13ePXoeT1zu39MIHwNkhp1qhKBDj1StZi9XEoO716p+qWT WJ9GomVPlCqQanMlW9vI0ExJukrKVmFfhoB/8GB9/cmFNbTsC/TfdGuMYQcNslsqKsy+rg jrUCDlpmLWVNMX7te0NhpZtdD47lVXkUrA1mchrDFqGgFPDaq0kG1QIleUBS0w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QpNtR0MfXzxYW; Sat, 24 Jun 2023 19:08:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 35OJ8UwE009065; Sat, 24 Jun 2023 19:08:30 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 35OJ8UjT009064; Sat, 24 Jun 2023 19:08:30 GMT (envelope-from git) Date: Sat, 24 Jun 2023 19:08:30 GMT Message-Id: <202306241908.35OJ8UjT009064@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alan Cox Subject: git: 0d2f98c2f092 - main - amd64 pmap: Tidy up pmap_promote_pde() calls List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: alc X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0d2f98c2f0928a8ee8446c3e5e0e4fb93f1dd9c5 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by alc: URL: https://cgit.FreeBSD.org/src/commit/?id=0d2f98c2f0928a8ee8446c3e5e0e4fb93f1dd9c5 commit 0d2f98c2f0928a8ee8446c3e5e0e4fb93f1dd9c5 Author: Alan Cox AuthorDate: 2023-06-17 17:18:33 +0000 Commit: Alan Cox CommitDate: 2023-06-24 18:09:04 +0000 amd64 pmap: Tidy up pmap_promote_pde() calls Since pmap_ps_enabled() is true by default, check it inside of pmap_promote_pde() instead of at every call site. Modify pmap_promote_pde() to return true if the promotion succeeded and false otherwise. Use this return value in a couple places. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D40744 --- sys/amd64/amd64/pmap.c | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 3cb02a4f9daa..3215a7f8d559 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -436,7 +436,7 @@ pt_entry_t pg_nx; static SYSCTL_NODE(_vm, OID_AUTO, pmap, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, "VM/pmap parameters"); -static int pg_ps_enabled = 1; +static int __read_frequently pg_ps_enabled = 1; SYSCTL_INT(_vm_pmap, OID_AUTO, pg_ps_enabled, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &pg_ps_enabled, 0, "Are large page mappings enabled?"); @@ -1318,7 +1318,7 @@ static void pmap_kenter_attr(vm_offset_t va, vm_paddr_t pa, int mode); static vm_page_t pmap_large_map_getptp_unlocked(void); static vm_paddr_t pmap_large_map_kextract(vm_offset_t va); #if VM_NRESERVLEVEL > 0 -static void pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, +static bool pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, vm_page_t mpte, struct rwlock **lockp); #endif static boolean_t pmap_protect_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t sva, @@ -6856,7 +6856,7 @@ pmap_pde_ept_executable(pmap_t pmap, pd_entry_t pde) * aligned, contiguous physical memory and (2) the 4KB page mappings must have * identical characteristics. */ -static void +static bool pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, vm_page_t mpte, struct rwlock **lockp) { @@ -6865,6 +6865,10 @@ pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, vm_page_t mpte, pt_entry_t allpte_PG_A, PG_A, PG_G, PG_M, PG_PKU_MASK, PG_RW, PG_V; int PG_PTE_CACHE; + PMAP_LOCK_ASSERT(pmap, MA_OWNED); + if (!pmap_ps_enabled(pmap)) + return (false); + PG_A = pmap_accessed_bit(pmap); PG_G = pmap_global_bit(pmap); PG_M = pmap_modified_bit(pmap); @@ -6873,8 +6877,6 @@ pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, vm_page_t mpte, PG_PKU_MASK = pmap_pku_mask_bit(pmap); PG_PTE_CACHE = pmap_cache_mask(pmap, 0); - PMAP_LOCK_ASSERT(pmap, MA_OWNED); - /* * Examine the first PTE in the specified PTP. Abort if this PTE is * ineligible for promotion due to hardware errata, invalid, or does @@ -6883,12 +6885,12 @@ pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, vm_page_t mpte, firstpte = (pt_entry_t *)PHYS_TO_DMAP(*pde & PG_FRAME); newpde = *firstpte; if (!pmap_allow_2m_x_page(pmap, pmap_pde_ept_executable(pmap, newpde))) - return; + return (false); if ((newpde & ((PG_FRAME & PDRMASK) | PG_V)) != PG_V) { counter_u64_add(pmap_pde_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_pde: failure for va %#lx" " in pmap %p", va, pmap); - return; + return (false); } /* @@ -6933,7 +6935,7 @@ setpde: counter_u64_add(pmap_pde_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_pde: failure for va %#lx" " in pmap %p", va, pmap); - return; + return (false); } setpte: if ((oldpte & (PG_M | PG_RW)) == PG_RW) { @@ -6952,7 +6954,7 @@ setpte: counter_u64_add(pmap_pde_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_pde: failure for va %#lx" " in pmap %p", va, pmap); - return; + return (false); } allpte_PG_A &= oldpte; pa -= PAGE_SIZE; @@ -6993,7 +6995,7 @@ setpte: CTR2(KTR_PMAP, "pmap_promote_pde: failure for va %#lx in pmap %p", va, pmap); - return; + return (false); } /* @@ -7018,6 +7020,7 @@ setpte: counter_u64_add(pmap_pde_promotions, 1); CTR2(KTR_PMAP, "pmap_promote_pde: success for va %#lx" " in pmap %p", va, pmap); + return (true); } #endif /* VM_NRESERVLEVEL > 0 */ @@ -7391,10 +7394,9 @@ unchanged: * populated, then attempt promotion. */ if ((mpte == NULL || mpte->ref_count == NPTEPG) && - pmap_ps_enabled(pmap) && (m->flags & PG_FICTITIOUS) == 0 && vm_reserv_level_iffullpop(m) == 0) - pmap_promote_pde(pmap, pde, va, mpte, &lock); + (void)pmap_promote_pde(pmap, pde, va, mpte, &lock); #endif rv = KERN_SUCCESS; @@ -7782,18 +7784,17 @@ pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, vm_page_t m, * attempt promotion. */ if ((mpte == NULL || mpte->ref_count == NPTEPG) && - pmap_ps_enabled(pmap) && (m->flags & PG_FICTITIOUS) == 0 && vm_reserv_level_iffullpop(m) == 0) { if (pde == NULL) pde = pmap_pde(pmap, va); - pmap_promote_pde(pmap, pde, va, mpte, lockp); /* * If promotion succeeds, then the next call to this function * should not be given the unmapped PTP as a hint. */ - mpte = NULL; + if (pmap_promote_pde(pmap, pde, va, mpte, lockp)) + mpte = NULL; } #endif @@ -10359,10 +10360,9 @@ pmap_emulate_accessed_dirty(pmap_t pmap, vm_offset_t va, int ftype) m = PHYS_TO_VM_PAGE(*pte & PG_FRAME); if ((mpte == NULL || mpte->ref_count == NPTEPG) && - pmap_ps_enabled(pmap) && (m->flags & PG_FICTITIOUS) == 0 && - vm_reserv_level_iffullpop(m) == 0) { - pmap_promote_pde(pmap, pde, va, mpte, &lock); + vm_reserv_level_iffullpop(m) == 0 && + pmap_promote_pde(pmap, pde, va, mpte, &lock)) { #ifdef INVARIANTS atomic_add_long(&ad_emulation_superpage_promotions, 1); #endif