From nobody Wed Oct 13 09:19:23 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 92A4E1800758; Wed, 13 Oct 2021 09:19:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HTn6P58h7z4skq; Wed, 13 Oct 2021 09:19:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 812DC25409; Wed, 13 Oct 2021 09:19:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 19D9JNeb057505; Wed, 13 Oct 2021 09:19:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 19D9JNWY057504; Wed, 13 Oct 2021 09:19:23 GMT (envelope-from git) Date: Wed, 13 Oct 2021 09:19:23 GMT Message-Id: <202110130919.19D9JNWY057504@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Vladimir Kondratyev Subject: git: e1f5bed753fe - stable/13 - LinuxKPI: Import linux_page.c and some dependent code from drm-kmod List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: wulf X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e1f5bed753fe6627f38ee3264f450679805e95da Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by wulf: URL: https://cgit.FreeBSD.org/src/commit/?id=e1f5bed753fe6627f38ee3264f450679805e95da commit e1f5bed753fe6627f38ee3264f450679805e95da Author: Vladimir Kondratyev AuthorDate: 2021-09-29 20:15:37 +0000 Commit: Vladimir Kondratyev CommitDate: 2021-10-13 09:07:53 +0000 LinuxKPI: Import linux_page.c and some dependent code from drm-kmod No functional changes intended Reviewed by: hselasky, manu, markj Differential revision: https://reviews.freebsd.org/D32167 (cherry picked from commit c072f6e856bc0348bf6fdd468761041948823f73) --- sys/compat/linuxkpi/common/include/linux/highmem.h | 118 +++++++++++++++++++++ sys/compat/linuxkpi/common/include/linux/mm.h | 4 + sys/compat/linuxkpi/common/include/linux/page.h | 22 ++++ .../linuxkpi/common/include/linux/scatterlist.h | 58 ++++++++++ sys/compat/linuxkpi/common/src/linux_page.c | 34 ++++++ 5 files changed, 236 insertions(+) diff --git a/sys/compat/linuxkpi/common/include/linux/highmem.h b/sys/compat/linuxkpi/common/include/linux/highmem.h new file mode 100644 index 000000000000..9378746ea480 --- /dev/null +++ b/sys/compat/linuxkpi/common/include/linux/highmem.h @@ -0,0 +1,118 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD + * + * Copyright (c) 2010 Isilon Systems, Inc. + * Copyright (c) 2016 Matthew Macy (mmacy@mattmacy.io) + * Copyright (c) 2017 Mellanox Technologies, Ltd. + * Copyright (c) 2021 Vladimir Kondratyev + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#ifndef _LINUX_HIGHMEM_H_ +#define _LINUX_HIGHMEM_H_ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include + +#define PageHighMem(p) (0) + +static inline vm_page_t +kmap_to_page(void *addr) +{ + return (virt_to_page(addr)); +} + +static inline void * +kmap(vm_page_t page) +{ + struct sf_buf *sf; + + if (PMAP_HAS_DMAP) { + return ((void *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(page))); + } else { + sched_pin(); + sf = sf_buf_alloc(page, SFB_NOWAIT | SFB_CPUPRIVATE); + if (sf == NULL) { + sched_unpin(); + return (NULL); + } + return ((void *)sf_buf_kva(sf)); + } +} + +static inline void * +kmap_atomic_prot(vm_page_t page, pgprot_t prot) +{ + vm_memattr_t attr = pgprot2cachemode(prot); + + if (attr != VM_MEMATTR_DEFAULT) { + vm_page_lock(page); + page->flags |= PG_FICTITIOUS; + vm_page_unlock(page); + pmap_page_set_memattr(page, attr); + } + return (kmap(page)); +} + +static inline void * +kmap_atomic(vm_page_t page) +{ + return (kmap_atomic_prot(page, VM_PROT_ALL)); +} + +static inline void +kunmap(vm_page_t page) +{ + struct sf_buf *sf; + + if (!PMAP_HAS_DMAP) { + /* lookup SF buffer in list */ + sf = sf_buf_alloc(page, SFB_NOWAIT | SFB_CPUPRIVATE); + + /* double-free */ + sf_buf_free(sf); + sf_buf_free(sf); + + sched_unpin(); + } +} + +static inline void +kunmap_atomic(void *vaddr) +{ + if (!PMAP_HAS_DMAP) + kunmap(virt_to_page(vaddr)); +} + +#endif /* _LINUX_HIGHMEM_H_ */ diff --git a/sys/compat/linuxkpi/common/include/linux/mm.h b/sys/compat/linuxkpi/common/include/linux/mm.h index 4301e616f234..0ee9dd58616a 100644 --- a/sys/compat/linuxkpi/common/include/linux/mm.h +++ b/sys/compat/linuxkpi/common/include/linux/mm.h @@ -290,4 +290,8 @@ vmalloc_to_page(const void *addr) extern int is_vmalloc_addr(const void *addr); void si_meminfo(struct sysinfo *si); +#define unmap_mapping_range(...) lkpi_unmap_mapping_range(__VA_ARGS__) +void lkpi_unmap_mapping_range(void *obj, loff_t const holebegin __unused, + loff_t const holelen, int even_cows __unused); + #endif /* _LINUX_MM_H_ */ diff --git a/sys/compat/linuxkpi/common/include/linux/page.h b/sys/compat/linuxkpi/common/include/linux/page.h index a09ae439266e..0c3d456d6ec0 100644 --- a/sys/compat/linuxkpi/common/include/linux/page.h +++ b/sys/compat/linuxkpi/common/include/linux/page.h @@ -41,6 +41,10 @@ #include #include +#if defined(__i386__) || defined(__amd64__) +#include +#endif + typedef unsigned long linux_pte_t; typedef unsigned long linux_pmd_t; typedef unsigned long linux_pgd_t; @@ -53,6 +57,8 @@ typedef unsigned long pgprot_t; CTASSERT((VM_PROT_ALL & -LINUXKPI_PROT_VALID) == 0); +#define PAGE_KERNEL_IO 0x0000 + static inline pgprot_t cachemode2protval(vm_memattr_t attr) { @@ -72,6 +78,7 @@ pgprot2cachemode(pgprot_t prot) #define page_to_pfn(pp) (VM_PAGE_TO_PHYS(pp) >> PAGE_SHIFT) #define pfn_to_page(pfn) (PHYS_TO_VM_PAGE((pfn) << PAGE_SHIFT)) #define nth_page(page,n) pfn_to_page(page_to_pfn(page) + (n)) +#define page_to_phys(page) VM_PAGE_TO_PHYS(page) #define clear_page(page) memset(page, 0, PAGE_SIZE) #define pgprot_noncached(prot) \ @@ -97,4 +104,19 @@ pgprot2cachemode(pgprot_t prot) #undef trunc_page #define trunc_page(x) ((uintptr_t)(x) & ~(PAGE_SIZE - 1)) +#if defined(__i386__) || defined(__amd64__) +#undef clflushopt +static inline void +lkpi_clflushopt(unsigned long addr) +{ + if (cpu_stdext_feature & CPUID_STDEXT_CLFLUSHOPT) + clflushopt(addr); + else if (cpu_feature & CPUID_CLFSH) + clflush(addr); + else + pmap_invalidate_cache(); +} +#define clflushopt(x) lkpi_clflushopt((unsigned long)(x)) +#endif + #endif /* _LINUX_PAGE_H_ */ diff --git a/sys/compat/linuxkpi/common/include/linux/scatterlist.h b/sys/compat/linuxkpi/common/include/linux/scatterlist.h index 5e42876facd0..13ee34cf9448 100644 --- a/sys/compat/linuxkpi/common/include/linux/scatterlist.h +++ b/sys/compat/linuxkpi/common/include/linux/scatterlist.h @@ -4,6 +4,7 @@ * Copyright (c) 2010 Panasas, Inc. * Copyright (c) 2013-2017 Mellanox Technologies, Ltd. * Copyright (c) 2015 Matthew Dillon + * Copyright (c) 2016 Matthew Macy * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -534,4 +535,61 @@ sg_pcopy_from_buffer(struct scatterlist *sgl, unsigned int nents, return (copied); } +static inline size_t +sg_copy_from_buffer(struct scatterlist *sgl, unsigned int nents, + const void *buf, size_t buflen) +{ + return (sg_pcopy_from_buffer(sgl, nents, buf, buflen, 0)); +} + +static inline size_t +sg_pcopy_to_buffer(struct scatterlist *sgl, unsigned int nents, + void *buf, size_t buflen, off_t offset) +{ + struct sg_page_iter iter; + struct scatterlist *sg; + struct page *page; + struct sf_buf *sf; + char *vaddr; + size_t total = 0; + size_t len; + + if (!PMAP_HAS_DMAP) + sched_pin(); + for_each_sg_page(sgl, &iter, nents, 0) { + sg = iter.sg; + + if (offset >= sg->length) { + offset -= sg->length; + continue; + } + len = ulmin(buflen, sg->length - offset); + if (len == 0) + break; + + page = sg_page_iter_page(&iter); + if (!PMAP_HAS_DMAP) { + sf = sf_buf_alloc(page, SFB_CPUPRIVATE | SFB_NOWAIT); + if (sf == NULL) + break; + vaddr = (char *)sf_buf_kva(sf); + } else + vaddr = (char *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(page)); + memcpy(buf, vaddr + sg->offset + offset, len); + if (!PMAP_HAS_DMAP) + sf_buf_free(sf); + + /* start at beginning of next page */ + offset = 0; + + /* advance buffer */ + buf = (char *)buf + len; + buflen -= len; + total += len; + } + if (!PMAP_HAS_DMAP) + sched_unpin(); + return (total); +} + #endif /* _LINUX_SCATTERLIST_H_ */ diff --git a/sys/compat/linuxkpi/common/src/linux_page.c b/sys/compat/linuxkpi/common/src/linux_page.c index c7191fef3429..8da4736ab7d5 100644 --- a/sys/compat/linuxkpi/common/src/linux_page.c +++ b/sys/compat/linuxkpi/common/src/linux_page.c @@ -334,3 +334,37 @@ retry: return (VM_FAULT_NOPAGE); } + +/* + * Although FreeBSD version of unmap_mapping_range has semantics and types of + * parameters compatible with Linux version, the values passed in are different + * @obj should match to vm_private_data field of vm_area_struct returned by + * mmap file operation handler, see linux_file_mmap_single() sources + * @holelen should match to size of area to be munmapped. + */ +void +lkpi_unmap_mapping_range(void *obj, loff_t const holebegin __unused, + loff_t const holelen, int even_cows __unused) +{ + vm_object_t devobj; + vm_page_t page; + int i, page_count; + + devobj = cdev_pager_lookup(obj); + if (devobj != NULL) { + page_count = OFF_TO_IDX(holelen); + + VM_OBJECT_WLOCK(devobj); +retry: + for (i = 0; i < page_count; i++) { + page = vm_page_lookup(devobj, i); + if (page == NULL) + continue; + if (!vm_page_busy_acquire(page, VM_ALLOC_WAITFAIL)) + goto retry; + cdev_pager_free_page(devobj, page); + } + VM_OBJECT_WUNLOCK(devobj); + vm_object_deallocate(devobj); + } +}