From nobody Wed Jun 05 04:28:10 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VvDwV5XQWz5LT68; Wed, 05 Jun 2024 04:28:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VvDwV33ncz4s4F; Wed, 5 Jun 2024 04:28:10 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1717561690; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/OHW7sH2YzOyfMTmJDPG5qFhULFHlFDAd8Vw2Ee1BhU=; b=C+WifYpHBhWf0SDKH0Ot8k0rlzzpy1p9roYXzk6yp83U0isjfTcu8U1vbVmNq376GYpB09 MqVQ3jtYs96gui8403P74Hl9rG9ypdX0kjy6/kkTDbMkCtcnbwMHaZSTok7q9fBPDCc78a etvKexUdZRvSpTyKExirloP4Os1IVaX/ia55cBUidWM3tFBlXHMKy36Ij7lMR6ICjOcyjK WyEJVk8ZkgFv3KxxhNPj1KKiw52U64X8qXfQiNfbV5exRTv9J18Nisb/v1hPCfcr61uoPh YliE0hqcDnZRTNk7+1Be5fZy80Y3c2QxxhDRDs+WpLxsOVdrbjoCepAyxkllxA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1717561690; a=rsa-sha256; cv=none; b=XHrMWkhiK46Km4J3fngoiQH865GpD3KefpHJIbOEuPP3WLDqc6zorCzCI9kZD+3Ao+HvJ+ x4gZTwI1sP0EZxx32QnCN06X0srnwKRaCkrd//aLirLwYYbgVmmT7uWp2XBYiQ41/8VXX8 HkYSa3tUomUJ429CqLFj2CpOot/P0auD2SFEtIh5DT+APF23A8RNgsD9L2f4+UUqO4au3N JFFj2GLJJcKJmcIDs3ovRm1uw5fXoYkY6IcLJ+HxL47qVVM54qvciHfk95KvkSkYVBjYrj 0tthX8I/sc0vFC10SFAuE5CJRcmZQrkGDe9JdVbfuUHkjlYo8qpbK7szwiHqXQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1717561690; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/OHW7sH2YzOyfMTmJDPG5qFhULFHlFDAd8Vw2Ee1BhU=; b=nzq+CvUqryK3PLQB14SjAYFESUE3pMJTI4diqvliUb9DdvzzrvF4Ht3EeJrQorMIIcMf/S XC7EXqNwLAEMD0LcH8d3yxk86or2NF/3+M4ln9oKt0AkuFH21Sx9wSlrDFoNGIizNgQcLY moQdEnz524PLMzqJsEc21PNeDICskNP6lOBBaKh8n3T1+5bZsrHmT8J9EbwqekGJKlfey8 anJlYKBKjW+VQbg5kh81BWe8847ZQbukkTJci2hdCUz0dHmYdoUecSzGWj9gwr5hZW28ln 6Se3lix6bP7Mkmyg5uV+cD4QS+mGMwFvKrKCb2Ah93UQIubDqh1p5cIEVdE6wA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VvDwV2TrNznd6; Wed, 5 Jun 2024 04:28:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 4554SAVr019895; Wed, 5 Jun 2024 04:28:10 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 4554SALi019892; Wed, 5 Jun 2024 04:28:10 GMT (envelope-from git) Date: Wed, 5 Jun 2024 04:28:10 GMT Message-Id: <202406050428.4554SALi019892@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alan Cox Subject: git: 41dfea24eec2 - main - arm64 pmap: Enable L3C promotions by pmap_enter_quick() List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: alc X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 41dfea24eec242e1e083e2a879483a7c05c7e2ff Auto-Submitted: auto-generated The branch main has been updated by alc: URL: https://cgit.FreeBSD.org/src/commit/?id=41dfea24eec242e1e083e2a879483a7c05c7e2ff commit 41dfea24eec242e1e083e2a879483a7c05c7e2ff Author: Alan Cox AuthorDate: 2024-06-01 18:17:52 +0000 Commit: Alan Cox CommitDate: 2024-06-05 04:25:51 +0000 arm64 pmap: Enable L3C promotions by pmap_enter_quick() More precisely, implement L3C (64KB/2MB, depending on base page size) promotion in pmap_enter_quick()'s helper function, pmap_enter_quick_locked(). At the same time, use the recently introduced flag VM_PROT_NO_PROMOTE from pmap_enter_object() to pmap_enter_quick_locked() to avoid L3C promotion attempts that will fail. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D45445 --- sys/arm64/arm64/pmap.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 03d0a1cc6676..8ac7b8f6a135 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -5883,9 +5883,19 @@ pmap_enter_object(pmap_t pmap, vm_offset_t start, vm_offset_t end, ((rv = pmap_enter_l3c_rx(pmap, va, m, &mpte, prot, &lock)) == KERN_SUCCESS || rv == KERN_NO_SPACE)) m = &m[L3C_ENTRIES - 1]; - else - mpte = pmap_enter_quick_locked(pmap, va, m, prot, mpte, - &lock); + else { + /* + * In general, if a superpage mapping were possible, + * it would have been created above. That said, if + * start and end are not superpage aligned, then + * promotion might be possible at the ends of [start, + * end). However, in practice, those promotion + * attempts are so unlikely to succeed that they are + * not worth trying. + */ + mpte = pmap_enter_quick_locked(pmap, va, m, prot | + VM_PROT_NO_PROMOTE, mpte, &lock); + } m = TAILQ_NEXT(m, listq); } if (lock != NULL) @@ -6048,12 +6058,19 @@ pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, vm_page_t m, #if VM_NRESERVLEVEL > 0 /* - * If both the PTP and the reservation are fully populated, then - * attempt promotion. + * First, attempt L3C promotion, if the virtual and physical addresses + * are aligned with each other and an underlying reservation has the + * neighboring L3 pages allocated. The first condition is simply an + * optimization that recognizes some eventual promotion failures early + * at a lower run-time cost. Then, attempt L2 promotion, if both the + * PTP and the reservation are fully populated. */ if ((prot & VM_PROT_NO_PROMOTE) == 0 && - (mpte == NULL || mpte->ref_count == NL3PG) && + (va & L3C_OFFSET) == (pa & L3C_OFFSET) && (m->flags & PG_FICTITIOUS) == 0 && + vm_reserv_is_populated(m, L3C_ENTRIES) && + pmap_promote_l3c(pmap, l3, va) && + (mpte == NULL || mpte->ref_count == NL3PG) && vm_reserv_level_iffullpop(m) == 0) { if (l2 == NULL) l2 = pmap_pde(pmap, va, &lvl);