From nobody Sat May 25 17:25:12 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Vmph833V1z5LDtY; Sat, 25 May 2024 17:25:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Vmph82NCqz4V3s; Sat, 25 May 2024 17:25:12 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716657912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=d0pWZosFQV1+zHG0ptnlb5++2Q0fKvrtLej6T0RzEa8=; b=Dgd9/RR0cbuGpC7Z2xuMKzYetmKApu7sExEIA9AEaQdv9lm6YKj3L5f561vlIKZpVnWuLa GqODqH1K5k0ErodueMA8HJasP1yJR2itr3TeeMsnk+fGkNJLKDaWAqgXZ0hrXQc6LNG/Vi wsktBZ0fi3a6jSVixFR9tolCZ6W0kRMEfeN/2SXVKuWC1JZnZ/N2fAgy7ez4mhKddtQtFR ToKI998J/6IkA93w9FbXVB4/2ZEbDekB18UfP6xBHgRboyOUj0DBj4/VdB+V9WCiyKWfnH 9rvRKMNdRR4EIFo9+4jvAkqW3fzdVoP8W/r5SUn0tRFQXhXC3WjiMD88wpsjpg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1716657912; a=rsa-sha256; cv=none; b=vEhQQRoZZfJ3qqFO0PBChAD+nEdiZ4kGTFEbBqyLS3n2IqzdNUvGSus9x1rxywzLPMO9Oe YF4nhLO8pOGtEmZNSKAZ1jpRUOmmAEuNKwbiXJ56jxURdpJJXAhHeZS0bcbHW129W1l4Ar TvIjCe/i61kN85x9h82GWaulQSex8/44snqJYnXaovsrdAPOELkS2JbehIyBpVmzjVveHZ sicLuspjDti0Pz6LGyDf0sBuycFuGd/AlDiwNGZkCII6dkAIAIabYj2tjNJ0Z6195tgtKX fNThck7rnfLzol13T7bpXLPqlJonTWFS4y+44raQ3RJ7h5EpCl6p/tp3KyLGwQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716657912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=d0pWZosFQV1+zHG0ptnlb5++2Q0fKvrtLej6T0RzEa8=; b=mwBmn/Uoy/+5UDnvmz4akmS0SdBihvZe2sCkioLWGibTNVMCFsx+hoSrChRe1g9v49XrUJ 6KyS6JmdMD5MK4MiOkUunmRdCo3xwXgaLUQCKkgr5r/pL3YNVxacdPvZ1atRJuVPZTw7Lq Htf4uD+utA0l24r21gdWW8g3YFwmYDLAO1VrYRJJqclcSrBktHj6YDynVp/YcS6AVdo+He xrnrYHoNfEhsHBwAF134pUxfXr7DR0LmauKDV+xR2ioh1MeleOYMbGKzc/gYDCysxGdT1H KTW4gOpSh0Ticx3LzU+VgWKw4pcSROxomf0ceJyE+hkioMOQ+/3Qxe6onVg2wQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Vmph81rWSzQqF; Sat, 25 May 2024 17:25:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 44PHPCuw026647; Sat, 25 May 2024 17:25:12 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 44PHPCSK026644; Sat, 25 May 2024 17:25:12 GMT (envelope-from git) Date: Sat, 25 May 2024 17:25:12 GMT Message-Id: <202405251725.44PHPCSK026644@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Bojan =?utf-8?Q?Novkovi=C4=87?= Subject: git: da76d349b6b1 - main - uma: Deduplicate uma_small_alloc List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: bnovkov X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: da76d349b6b104f4e70562304c800a0793dea18d Auto-Submitted: auto-generated The branch main has been updated by bnovkov: URL: https://cgit.FreeBSD.org/src/commit/?id=da76d349b6b104f4e70562304c800a0793dea18d commit da76d349b6b104f4e70562304c800a0793dea18d Author: Bojan Novković AuthorDate: 2024-05-03 16:48:18 +0000 Commit: Bojan Novković CommitDate: 2024-05-25 17:24:46 +0000 uma: Deduplicate uma_small_alloc This commit refactors the UMA small alloc code and removes most UMA machine-dependent code. The existing machine-dependent uma_small_alloc code is almost identical across all architectures, except for powerpc where using the direct map addresses involved extra steps in some cases. The MI/MD split was replaced by a default uma_small_alloc implementation that can be overridden by architecture-specific code by defining the UMA_MD_SMALL_ALLOC symbol. Furthermore, UMA_USE_DMAP was introduced to replace most UMA_MD_SMALL_ALLOC uses. Reviewed by: markj, kib Approved by: markj (mentor) Differential Revision: https://reviews.freebsd.org/D45084 --- sys/amd64/amd64/uma_machdep.c | 71 ------------------------------------------- sys/amd64/include/vmparam.h | 6 ++-- sys/arm64/arm64/uma_machdep.c | 69 ----------------------------------------- sys/arm64/include/vmparam.h | 2 +- sys/conf/files.amd64 | 1 - sys/conf/files.arm64 | 1 - sys/conf/files.riscv | 1 - sys/kern/subr_vmem.c | 6 ++-- sys/powerpc/include/vmparam.h | 6 ++-- sys/riscv/include/vmparam.h | 2 +- sys/riscv/riscv/uma_machdep.c | 68 ----------------------------------------- sys/vm/uma_core.c | 43 ++++++++++++++++++++++++-- sys/vm/vm_map.c | 8 ++--- sys/vm/vm_radix.c | 2 +- 14 files changed, 57 insertions(+), 229 deletions(-) diff --git a/sys/amd64/amd64/uma_machdep.c b/sys/amd64/amd64/uma_machdep.c deleted file mode 100644 index f83f0674cc4e..000000000000 --- a/sys/amd64/amd64/uma_machdep.c +++ /dev/null @@ -1,71 +0,0 @@ -/*- - * SPDX-License-Identifier: BSD-2-Clause - * - * Copyright (c) 2003 Alan L. Cox - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -void * -uma_small_alloc(uma_zone_t zone, vm_size_t bytes, int domain, u_int8_t *flags, - int wait) -{ - vm_page_t m; - vm_paddr_t pa; - void *va; - - *flags = UMA_SLAB_PRIV; - m = vm_page_alloc_noobj_domain(domain, malloc2vm_flags(wait) | - VM_ALLOC_WIRED); - if (m == NULL) - return (NULL); - pa = m->phys_addr; - if ((wait & M_NODUMP) == 0) - dump_add_page(pa); - va = (void *)PHYS_TO_DMAP(pa); - return (va); -} - -void -uma_small_free(void *mem, vm_size_t size, u_int8_t flags) -{ - vm_page_t m; - vm_paddr_t pa; - - pa = DMAP_TO_PHYS((vm_offset_t)mem); - dump_drop_page(pa); - m = PHYS_TO_VM_PAGE(pa); - vm_page_unwire_noq(m); - vm_page_free(m); -} diff --git a/sys/amd64/include/vmparam.h b/sys/amd64/include/vmparam.h index bff9bf840036..e5155a7c7d47 100644 --- a/sys/amd64/include/vmparam.h +++ b/sys/amd64/include/vmparam.h @@ -72,12 +72,12 @@ #endif /* - * We provide a machine specific single page allocator through the use - * of the direct mapped segment. This uses 2MB pages for reduced + * We provide a single page allocator through the use of the + * direct mapped segment. This uses 2MB pages for reduced * TLB pressure. */ #if !defined(KASAN) && !defined(KMSAN) -#define UMA_MD_SMALL_ALLOC +#define UMA_USE_DMAP #endif /* diff --git a/sys/arm64/arm64/uma_machdep.c b/sys/arm64/arm64/uma_machdep.c deleted file mode 100644 index f942248d4dcd..000000000000 --- a/sys/arm64/arm64/uma_machdep.c +++ /dev/null @@ -1,69 +0,0 @@ -/*- - * Copyright (c) 2003 Alan L. Cox - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -void * -uma_small_alloc(uma_zone_t zone, vm_size_t bytes, int domain, u_int8_t *flags, - int wait) -{ - vm_page_t m; - vm_paddr_t pa; - void *va; - - *flags = UMA_SLAB_PRIV; - m = vm_page_alloc_noobj_domain(domain, malloc2vm_flags(wait) | - VM_ALLOC_WIRED); - if (m == NULL) - return (NULL); - pa = m->phys_addr; - if ((wait & M_NODUMP) == 0) - dump_add_page(pa); - va = (void *)PHYS_TO_DMAP(pa); - return (va); -} - -void -uma_small_free(void *mem, vm_size_t size, u_int8_t flags) -{ - vm_page_t m; - vm_paddr_t pa; - - pa = DMAP_TO_PHYS((vm_offset_t)mem); - dump_drop_page(pa); - m = PHYS_TO_VM_PAGE(pa); - vm_page_unwire_noq(m); - vm_page_free(m); -} diff --git a/sys/arm64/include/vmparam.h b/sys/arm64/include/vmparam.h index ffa5a538504a..0dcd02d63938 100644 --- a/sys/arm64/include/vmparam.h +++ b/sys/arm64/include/vmparam.h @@ -293,7 +293,7 @@ #endif #if !defined(KASAN) && !defined(KMSAN) -#define UMA_MD_SMALL_ALLOC +#define UMA_USE_DMAP #endif #ifndef LOCORE diff --git a/sys/conf/files.amd64 b/sys/conf/files.amd64 index 18dec5ed47b0..add27418ce08 100644 --- a/sys/conf/files.amd64 +++ b/sys/conf/files.amd64 @@ -92,7 +92,6 @@ amd64/amd64/support.S standard amd64/amd64/sys_machdep.c standard amd64/amd64/trap.c standard amd64/amd64/uio_machdep.c standard -amd64/amd64/uma_machdep.c standard amd64/amd64/vm_machdep.c standard amd64/pci/pci_cfgreg.c optional pci cddl/dev/dtrace/amd64/dtrace_asm.S optional dtrace compile-with "${DTRACE_S}" diff --git a/sys/conf/files.arm64 b/sys/conf/files.arm64 index a3d4fc09da89..8139a7af8ed3 100644 --- a/sys/conf/files.arm64 +++ b/sys/conf/files.arm64 @@ -78,7 +78,6 @@ arm64/arm64/swtch.S standard arm64/arm64/sys_machdep.c standard arm64/arm64/trap.c standard arm64/arm64/uio_machdep.c standard -arm64/arm64/uma_machdep.c standard arm64/arm64/undefined.c standard arm64/arm64/unwind.c optional ddb | kdtrace_hooks | stack \ compile-with "${NORMAL_C:N-fsanitize*:N-fno-sanitize*}" diff --git a/sys/conf/files.riscv b/sys/conf/files.riscv index be7ae2b40a08..49c8ddd0c516 100644 --- a/sys/conf/files.riscv +++ b/sys/conf/files.riscv @@ -67,7 +67,6 @@ riscv/riscv/sys_machdep.c standard riscv/riscv/trap.c standard riscv/riscv/timer.c standard riscv/riscv/uio_machdep.c standard -riscv/riscv/uma_machdep.c standard riscv/riscv/unwind.c optional ddb | kdtrace_hooks | stack riscv/riscv/vm_machdep.c standard diff --git a/sys/kern/subr_vmem.c b/sys/kern/subr_vmem.c index 1c9a8a5be979..a706d944dc3f 100644 --- a/sys/kern/subr_vmem.c +++ b/sys/kern/subr_vmem.c @@ -624,14 +624,14 @@ qc_drain(vmem_t *vm) uma_zone_reclaim(vm->vm_qcache[i].qc_cache, UMA_RECLAIM_DRAIN); } -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP static struct mtx_padalign __exclusive_cache_line vmem_bt_lock; /* * vmem_bt_alloc: Allocate a new page of boundary tags. * - * On architectures with uma_small_alloc there is no recursion; no address + * On architectures with UMA_USE_DMAP there is no recursion; no address * space need be allocated to allocate boundary tags. For the others, we * must handle recursion. Boundary tags are necessary to allocate new * boundary tags. @@ -707,7 +707,7 @@ vmem_startup(void) vmem_bt_zone = uma_zcreate("vmem btag", sizeof(struct vmem_btag), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_VM); -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP mtx_init(&vmem_bt_lock, "btag lock", NULL, MTX_DEF); uma_prealloc(vmem_bt_zone, BT_MAXALLOC); /* diff --git a/sys/powerpc/include/vmparam.h b/sys/powerpc/include/vmparam.h index 89982a618bc7..250da8298610 100644 --- a/sys/powerpc/include/vmparam.h +++ b/sys/powerpc/include/vmparam.h @@ -111,6 +111,8 @@ #define KERNBASE 0x00100100 /* start of kernel virtual */ +#define UMA_MD_SMALL_ALLOC + #ifdef AIM #ifndef __powerpc64__ #define VM_MIN_KERNEL_ADDRESS ((vm_offset_t)KERNEL_SR << ADDR_SR_SHFT) @@ -122,13 +124,13 @@ * Use the direct-mapped BAT registers for UMA small allocs. This * takes pressure off the small amount of available KVA. */ -#define UMA_MD_SMALL_ALLOC +#define UMA_USE_DMAP #else /* Book-E */ /* Use the direct map for UMA small allocs on powerpc64. */ #ifdef __powerpc64__ -#define UMA_MD_SMALL_ALLOC +#define UMA_USE_DMAP #else #define VM_MIN_KERNEL_ADDRESS 0xc0000000 #define VM_MAX_KERNEL_ADDRESS 0xffffefff diff --git a/sys/riscv/include/vmparam.h b/sys/riscv/include/vmparam.h index d2014654b691..5711bc8c347e 100644 --- a/sys/riscv/include/vmparam.h +++ b/sys/riscv/include/vmparam.h @@ -234,7 +234,7 @@ #define VM_INITIAL_PAGEIN 16 #endif -#define UMA_MD_SMALL_ALLOC +#define UMA_USE_DMAP #ifndef LOCORE extern vm_paddr_t dmap_phys_base; diff --git a/sys/riscv/riscv/uma_machdep.c b/sys/riscv/riscv/uma_machdep.c deleted file mode 100644 index 54e0d25800f6..000000000000 --- a/sys/riscv/riscv/uma_machdep.c +++ /dev/null @@ -1,68 +0,0 @@ -/*- - * Copyright (c) 2003 Alan L. Cox - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -void * -uma_small_alloc(uma_zone_t zone, vm_size_t bytes, int domain, u_int8_t *flags, - int wait) -{ - vm_page_t m; - vm_paddr_t pa; - void *va; - - *flags = UMA_SLAB_PRIV; - m = vm_page_alloc_noobj_domain(domain, malloc2vm_flags(wait) | - VM_ALLOC_WIRED); - if (m == NULL) - return (NULL); - pa = m->phys_addr; - if ((wait & M_NODUMP) == 0) - dump_add_page(pa); - va = (void *)PHYS_TO_DMAP(pa); - return (va); -} - -void -uma_small_free(void *mem, vm_size_t size, u_int8_t flags) -{ - vm_page_t m; - vm_paddr_t pa; - - pa = DMAP_TO_PHYS((vm_offset_t)mem); - dump_drop_page(pa); - m = PHYS_TO_VM_PAGE(pa); - vm_page_unwire_noq(m); - vm_page_free(m); -} diff --git a/sys/vm/uma_core.c b/sys/vm/uma_core.c index d185f12448ee..f9b6e18899c6 100644 --- a/sys/vm/uma_core.c +++ b/sys/vm/uma_core.c @@ -2079,6 +2079,28 @@ contig_alloc(uma_zone_t zone, vm_size_t bytes, int domain, uint8_t *pflag, bytes, wait, 0, ~(vm_paddr_t)0, 1, 0, VM_MEMATTR_DEFAULT)); } +#if defined(UMA_USE_DMAP) && !defined(UMA_MD_SMALL_ALLOC) +void * +uma_small_alloc(uma_zone_t zone, vm_size_t bytes, int domain, uint8_t *flags, + int wait) +{ + vm_page_t m; + vm_paddr_t pa; + void *va; + + *flags = UMA_SLAB_PRIV; + m = vm_page_alloc_noobj_domain(domain, + malloc2vm_flags(wait) | VM_ALLOC_WIRED); + if (m == NULL) + return (NULL); + pa = m->phys_addr; + if ((wait & M_NODUMP) == 0) + dump_add_page(pa); + va = (void *)PHYS_TO_DMAP(pa); + return (va); +} +#endif + /* * Frees a number of pages to the system * @@ -2141,6 +2163,21 @@ pcpu_page_free(void *mem, vm_size_t size, uint8_t flags) kva_free(sva, size); } +#if defined(UMA_USE_DMAP) && !defined(UMA_MD_SMALL_ALLOC) +void +uma_small_free(void *mem, vm_size_t size, uint8_t flags) +{ + vm_page_t m; + vm_paddr_t pa; + + pa = DMAP_TO_PHYS((vm_offset_t)mem); + dump_drop_page(pa); + m = PHYS_TO_VM_PAGE(pa); + vm_page_unwire_noq(m); + vm_page_free(m); +} +#endif + /* * Zero fill initializer * @@ -3154,7 +3191,7 @@ uma_startup1(vm_offset_t virtual_avail) smr_init(); } -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP extern void vm_radix_reserve_kva(void); #endif @@ -3174,7 +3211,7 @@ uma_startup2(void) vm_map_unlock(kernel_map); } -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP /* Set up radix zone to use noobj_alloc. */ vm_radix_reserve_kva(); #endif @@ -5171,7 +5208,7 @@ uma_zone_reserve_kva(uma_zone_t zone, int count) pages = howmany(count, keg->uk_ipers) * keg->uk_ppera; -#ifdef UMA_MD_SMALL_ALLOC +#ifdef UMA_USE_DMAP if (keg->uk_ppera > 1) { #else if (1) { diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c index 3111dda6e99d..3c7afcb6642f 100644 --- a/sys/vm/vm_map.c +++ b/sys/vm/vm_map.c @@ -172,7 +172,7 @@ static void vm_map_wire_entry_failure(vm_map_t map, vm_map_entry_t entry, start = end; \ } -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP /* * Allocate a new slab for kernel map entries. The kernel map may be locked or @@ -264,7 +264,7 @@ vm_map_startup(void) kmapentzone = uma_zcreate("KMAP ENTRY", sizeof(struct vm_map_entry), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_VM | UMA_ZONE_NOBUCKET); -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP /* Reserve an extra map entry for use when replenishing the reserve. */ uma_zone_reserve(kmapentzone, KMAPENT_RESERVE + 1); uma_prealloc(kmapentzone, KMAPENT_RESERVE + 1); @@ -660,7 +660,7 @@ _vm_map_unlock(vm_map_t map, const char *file, int line) VM_MAP_UNLOCK_CONSISTENT(map); if (map->system_map) { -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP if (map == kernel_map && (map->flags & MAP_REPLENISH) != 0) { uma_prealloc(kmapentzone, 1); map->flags &= ~MAP_REPLENISH; @@ -937,7 +937,7 @@ vm_map_entry_create(vm_map_t map) { vm_map_entry_t new_entry; -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP if (map == kernel_map) { VM_MAP_ASSERT_LOCKED(map); diff --git a/sys/vm/vm_radix.c b/sys/vm/vm_radix.c index cfc5a82eacc8..13f9d36194ab 100644 --- a/sys/vm/vm_radix.c +++ b/sys/vm/vm_radix.c @@ -82,7 +82,7 @@ vm_radix_node_free(struct pctrie *ptree, void *node) uma_zfree_smr(vm_radix_node_zone, node); } -#ifndef UMA_MD_SMALL_ALLOC +#ifndef UMA_USE_DMAP void vm_radix_reserve_kva(void); /* * Reserve the KVA necessary to satisfy the node allocation.