From nobody Thu Nov 11 19:52:28 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 07A13185B703; Thu, 11 Nov 2021 19:52:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HqsnS6ndPz4gX2; Thu, 11 Nov 2021 19:52:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C8D25123A9; Thu, 11 Nov 2021 19:52:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1ABJqSZi089932; Thu, 11 Nov 2021 19:52:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1ABJqSMu089931; Thu, 11 Nov 2021 19:52:28 GMT (envelope-from git) Date: Thu, 11 Nov 2021 19:52:28 GMT Message-Id: <202111111952.1ABJqSMu089931@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: e4bdb6857aea - main - vm_page: Handle VM_ALLOC_NORECLAIM in the contiguous page allocator List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e4bdb6857aea466ce4af48c0bd05a214460fe313 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=e4bdb6857aea466ce4af48c0bd05a214460fe313 commit e4bdb6857aea466ce4af48c0bd05a214460fe313 Author: Mark Johnston AuthorDate: 2021-11-11 19:26:41 +0000 Commit: Mark Johnston CommitDate: 2021-11-11 19:26:41 +0000 vm_page: Handle VM_ALLOC_NORECLAIM in the contiguous page allocator We added _NORECLAIM to request that kmem_alloc_contig_pages() not spend time scanning physical memory for candidates to reclaim. In some situations the scanning can induce large amounts of undesirable latency, and it's less important that the request be satisfied than it is that we not spend many milliseconds scanning. The problem extends to vm_reserv_reclaim_contig(), which unlike vm_reserv_reclaim() may have to scan the entire list of partially populated reservations. Use VM_ALLOC_NORECLAIM to request that this scan not be executed.[1] As a side effect, this fixes a regression in 02fb0585e7b3 ("vm_page: Drop handling of VM_ALLOC_NOOBJ in vm_page_alloc_contig_domain()") where VM_ALLOC_CONTIG was not included in VPAC_FLAGS or VPANC_FLAGS even though it is not masked by kmem_alloc_contig_pages().[2] Reported by: gallatin [1], glebius [2] Reviewed by: alc, glebius, kib MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D32899 --- sys/vm/vm_kern.c | 4 ---- sys/vm/vm_page.c | 16 ++++++++++++---- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/sys/vm/vm_kern.c b/sys/vm/vm_kern.c index 4ddeb3208260..669dc194759b 100644 --- a/sys/vm/vm_kern.c +++ b/sys/vm/vm_kern.c @@ -182,10 +182,6 @@ kmem_alloc_contig_pages(vm_object_t object, vm_pindex_t pindex, int domain, VM_OBJECT_ASSERT_WLOCKED(object); - /* Disallow an invalid combination of flags. */ - MPASS((pflags & (VM_ALLOC_WAITOK | VM_ALLOC_NORECLAIM)) != - (VM_ALLOC_WAITOK | VM_ALLOC_NORECLAIM)); - wait = (pflags & VM_ALLOC_WAITOK) != 0; reclaim = (pflags & VM_ALLOC_NORECLAIM) == 0; pflags &= ~(VM_ALLOC_NOWAIT | VM_ALLOC_WAITOK | VM_ALLOC_WAITFAIL); diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c index 76e9ba4db403..ac710bf43d08 100644 --- a/sys/vm/vm_page.c +++ b/sys/vm/vm_page.c @@ -2250,12 +2250,15 @@ vm_page_alloc_contig_domain(vm_object_t object, vm_pindex_t pindex, int domain, vm_page_t m, m_ret, mpred; u_int busy_lock, flags, oflags; -#define VPAC_FLAGS VPA_FLAGS +#define VPAC_FLAGS (VPA_FLAGS | VM_ALLOC_NORECLAIM) KASSERT((req & ~VPAC_FLAGS) == 0, ("invalid request %#x", req)); KASSERT(((req & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) != (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)), ("invalid request %#x", req)); + KASSERT((req & (VM_ALLOC_WAITOK | VM_ALLOC_NORECLAIM)) != + (VM_ALLOC_WAITOK | VM_ALLOC_NORECLAIM), + ("invalid request %#x", req)); VM_OBJECT_ASSERT_WLOCKED(object); KASSERT((object->flags & OBJ_FICTITIOUS) == 0, ("vm_page_alloc_contig: object %p has fictitious pages", @@ -2294,7 +2297,8 @@ again: if (m_ret == NULL) { vm_domain_freecnt_inc(vmd, npages); #if VM_NRESERVLEVEL > 0 - if (vm_reserv_reclaim_contig(domain, npages, low, + if ((req & VM_ALLOC_NORECLAIM) == 0 && + vm_reserv_reclaim_contig(domain, npages, low, high, alignment, boundary)) goto again; #endif @@ -2525,9 +2529,12 @@ vm_page_alloc_noobj_contig_domain(int domain, int req, u_long npages, vm_page_t m, m_ret; u_int flags; -#define VPANC_FLAGS VPAN_FLAGS +#define VPANC_FLAGS (VPAN_FLAGS | VM_ALLOC_NORECLAIM) KASSERT((req & ~VPANC_FLAGS) == 0, ("invalid request %#x", req)); + KASSERT((req & (VM_ALLOC_WAITOK | VM_ALLOC_NORECLAIM)) != + (VM_ALLOC_WAITOK | VM_ALLOC_NORECLAIM), + ("invalid request %#x", req)); KASSERT(((req & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) != (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)), ("invalid request %#x", req)); @@ -2547,7 +2554,8 @@ again: if (m_ret == NULL) { vm_domain_freecnt_inc(vmd, npages); #if VM_NRESERVLEVEL > 0 - if (vm_reserv_reclaim_contig(domain, npages, low, + if ((req & VM_ALLOC_NORECLAIM) == 0 && + vm_reserv_reclaim_contig(domain, npages, low, high, alignment, boundary)) goto again; #endif