From nobody Wed Aug 17 19:11:08 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M7Hg03BzZz4Z1N9; Wed, 17 Aug 2022 19:11:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M7Hg02hcxz4LQV; Wed, 17 Aug 2022 19:11:08 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1660763468; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gH+MmirgM7JoH6CQIBA0QEjJqPJZZyxqr77fQSb9nKw=; b=vBLg1MnwkfBz8W2o5pb12ighJFlIZtRz021m3wSRjW7clYF/7lZDFQEOcBllHa7rbO0jMW e2QDdHIS3u9+Sj9KYRusE6YsUN5SNysCWv3q/Bh3Qcof+mrGzVaNouocZ7WX/I/C9qyBzD CEzunbx/cQ/rspilstVcHhkgmsrwXoSuhPldBxnkJcff0E6P3FpnRA98SgVFiGDvwv9JQs InASsgGoFpjjJ1PdBzEs8mSCO8kzJfiOAtAAiApsSX7Lf4tedvF5dyioCArPHzaODKgKFI 0/7Afpgz2/76WCeUpDHYyFY1GHRH3O3Dr9yNQIQdsflM6pDpWGT4/KECgNyDUA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4M7Hg01mXMz11Q6; Wed, 17 Aug 2022 19:11:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 27HJB8Lj040770; Wed, 17 Aug 2022 19:11:08 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 27HJB8ls040769; Wed, 17 Aug 2022 19:11:08 GMT (envelope-from git) Date: Wed, 17 Aug 2022 19:11:08 GMT Message-Id: <202208171911.27HJB8ls040769@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 3499df29c2a1 - main - arm64 pmap: Convert PC_IS_FREE into an inline function. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 3499df29c2a1bbd6755f4ebe4b4a0083fda7a666 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1660763468; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gH+MmirgM7JoH6CQIBA0QEjJqPJZZyxqr77fQSb9nKw=; b=q4+dqixzH+uUofrmDe1on3x2Lxy5W4cwZAvMKG+cuPG9ldW9XAce46oK3TJsWI5AA4fEtK UWmF3lRXE1C7KxmjGdnAZTjqgaq+bqgos0gjmkqJLvr8gzLwzzKq9F8s/9sd7zybvkw0Io mLRR4SsJPwf4it/C80KSa2EZcwL0TCpBViRdiGTfmhC6zea/fUyv5m4dZKW3uNpXn/3TRl /uN/xUI//kZMvBtAGMwxabPpFCt1ji7GUGVjg1K1IvRF9iKde3BbJkBft1bhBq4ZsPm+wA YGxhSrctzdwV/UM9JbyTu9j13BgnQTNPE3ykZDcRIdCahKKV5n1aJx7B/JfMZw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1660763468; a=rsa-sha256; cv=none; b=F/o7updl9NQhT2oMS6akjb3KDUbiwF5hmtHJHHJ0jkMSpLVh/aOT+ad+PJelvmojnKch9T WtnfVmiXKGRxjG/HESY7uzfLzAWNGp22YNbKTIW796UJYff3HRz7h3Y3GgvUlR32J6ImzA xFgRcnWM68oD07SEPpA7jJMtTCCB6HPekEfiq0NRG9ZeB8sAsvf5M3HPjWVzAiooiNGrc4 OVLoMlDIPBfdvQnCCyXIPQO0lU6xIC37i4TZSHky0Cs14CNdOV+h8oimP486F4NVx9My49 Ffkj7DXYEjA+sp5cJ1A4nRge3u2Spa84JfYrBUyfzpaGJf43It45gCITDQDYbQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=3499df29c2a1bbd6755f4ebe4b4a0083fda7a666 commit 3499df29c2a1bbd6755f4ebe4b4a0083fda7a666 Author: John Baldwin AuthorDate: 2022-08-17 19:10:35 +0000 Commit: John Baldwin CommitDate: 2022-08-17 19:10:35 +0000 arm64 pmap: Convert PC_IS_FREE into an inline function. This permits inlining the comparisons even in the 16K page case. Note that since PC_FREEN is -1, values can be compared efficiently without having to fetch words of pc_freemask from memory via the 'cmn , #0x1' instruction. Reviewed by: markj Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D36218 --- sys/arm64/arm64/pmap.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 9c47c043d251..deea00bc5d13 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -2462,14 +2462,6 @@ pv_to_chunk(pv_entry_t pv) #define PC_FREEN 0xfffffffffffffffful #define PC_FREEL ((1ul << (_NPCPV % 64)) - 1) -#if _NPCM == 3 -#define PC_IS_FREE(pc) ((pc)->pc_map[0] == PC_FREEN && \ - (pc)->pc_map[1] == PC_FREEN && (pc)->pc_map[2] == PC_FREEL) -#else -#define PC_IS_FREE(pc) \ - (memcmp((pc)->pc_map, pc_freemask, sizeof(pc_freemask)) == 0) -#endif - static const uint64_t pc_freemask[] = { PC_FREEN, PC_FREEN, #if _NPCM > 3 PC_FREEN, PC_FREEN, PC_FREEN, PC_FREEN, PC_FREEN, PC_FREEN, PC_FREEN, @@ -2489,6 +2481,15 @@ pc_is_full(struct pv_chunk *pc) return (true); } +static __inline bool +pc_is_free(struct pv_chunk *pc) +{ + for (u_int i = 0; i < _NPCM - 1; i++) + if (pc->pc_map[i] != PC_FREEN) + return (false); + return (pc->pc_map[_NPCM - 1] == PC_FREEL); +} + #ifdef PV_STATS static int pc_chunk_count, pc_chunk_allocs, pc_chunk_frees, pc_chunk_tryfail; @@ -2653,7 +2654,7 @@ reclaim_pv_chunk(pmap_t locked_pmap, struct rwlock **lockp) PV_STAT(atomic_add_int(&pv_entry_spare, freed)); PV_STAT(atomic_subtract_long(&pv_entry_count, freed)); TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); - if (PC_IS_FREE(pc)) { + if (pc_is_free(pc)) { PV_STAT(atomic_subtract_int(&pv_entry_spare, _NPCPV)); PV_STAT(atomic_subtract_int(&pc_chunk_count, 1)); PV_STAT(atomic_add_int(&pc_chunk_frees, 1)); @@ -2722,7 +2723,7 @@ free_pv_entry(pmap_t pmap, pv_entry_t pv) field = idx / 64; bit = idx % 64; pc->pc_map[field] |= 1ul << bit; - if (!PC_IS_FREE(pc)) { + if (!pc_is_free(pc)) { /* 98% of the time, pc is already at the head of the list. */ if (__predict_false(pc != TAILQ_FIRST(&pmap->pm_pvchunk))) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list);