From nobody Sat Feb 10 22:03:23 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TXPqc2yTzz5BRmT; Sat, 10 Feb 2024 22:03:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TXPqc1864z4gJp; Sat, 10 Feb 2024 22:03:24 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707602604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=t4VM4rR4WSRUrcJI1gJGPUvj1JaBhapW9efC960x1ko=; b=y1rsj87nbPDrZx5/hloPMXZzj63ZwYkOZHmurhs54kvazr/1jOgjyr1g62Kta9iUzfxKF8 SzxIpG9mNxlrnhujuA217i62rcYT4prebu6wsFMq3HHYaXMsZVcSuVPkJdUoamSZFmkfsY T0m7k52yC7flChMhspLpbQgv0pjpWc/qWfqAjXuKx48ohYX5ar636HdE7e75H7YzKi0ef1 EuT4B8yi+w4JVAKCOHkiMU9SnzG7fdzOOn/alt4mVrSdDBaoqUz2uWyOP5ND5BRLRXzaOE eEKUonxFNkzCUlAKw54EXcqNgsDs403XSjW7I8fliYa4DFETO7/rvsB3YSwSvw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707602604; a=rsa-sha256; cv=none; b=KeKF8dk+DGBnQc1bi6Dicr9UZZ/Nrqa4LqIIws2xiQDxvYbGuFhTAFGcEkVupezZJCjPZU g17ihYanS0/NPh2N71qkw+0V/y2jRSG3Jnl32452rPEWjzJ+Q7UaVcZ2HJJWYmB4ObPtvJ jd9m7jHw15Hgv878ERZ48PUx/CO8VQ/XSsHI6oYQ11ACBIjQQTmHjHqSP+236wiy1/RE2Y ymZD60zzd7ZUGsQRyOufYXY0KwnViQtLsLJOL4m3bZzVc4675NnQmD23VIk+6Tb+3hZMJ4 nuQYceDrpjubVlM3fevvx1ZyKJR9yFpG8SpLSm4K+wxhvbg7vAQvZW8RdLaTRw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707602604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=t4VM4rR4WSRUrcJI1gJGPUvj1JaBhapW9efC960x1ko=; b=EZhfn/mvKK75Jem1i0vefePCQeuW0TpKBpPoYGbgoJeBfWQea+fldED40xsnQUXk6IQZgP 1ymlY+mITO3+GB4ea5TKoKhC+FGCi9CyWcbqQXGmhCkPkVf5T2VShaRwOJ8wg5QvC9jMvE +k5vrXGU7KwBaqlCPVOpFuC2nI+zlaHHOme+XObLYWss7duZ3FIhpHmDwVeqeBSiSHZrIn ldhV+Iji/hmRou4LB9ZSo/G7D7iI14KZ74zAifcRfStBsqbTf1x1V/08QCwrDUS3ydDSvO HQwv1mhuzRWKtXcDDp14CN2DNTiMrVl+V9JIxpoJZwFVF30BUM08tB8j7IqFJg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TXPqc0F9mzZHG; Sat, 10 Feb 2024 22:03:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 41AM3NRf006026; Sat, 10 Feb 2024 22:03:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 41AM3NGb006023; Sat, 10 Feb 2024 22:03:23 GMT (envelope-from git) Date: Sat, 10 Feb 2024 22:03:23 GMT Message-Id: <202402102203.41AM3NGb006023@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Vladimir Kondratyev Subject: git: c0b8047bdc13 - main - LinuxKPI: Allow kmalloc to be called when FPU protection is enabled List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: wulf X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: c0b8047bdc13040eafb162c4b7b5dba11034ff4b Auto-Submitted: auto-generated The branch main has been updated by wulf: URL: https://cgit.FreeBSD.org/src/commit/?id=c0b8047bdc13040eafb162c4b7b5dba11034ff4b commit c0b8047bdc13040eafb162c4b7b5dba11034ff4b Author: Vladimir Kondratyev AuthorDate: 2024-02-10 22:01:50 +0000 Commit: Vladimir Kondratyev CommitDate: 2024-02-10 22:01:50 +0000 LinuxKPI: Allow kmalloc to be called when FPU protection is enabled Amdgpu driver does a lot of memory allocations in FPU-protected sections of code for certain display cores, e.g. for DCN30. This does not work on FreeBSD as its malloc function can not be run within a critical section. Check this condition and temporally exit from FPU-protected context to workaround issue and reduce source code patching. Sponsored by: Serenity Cyber Security, LLC Reviewed by: manu (previous version) MFC after: 1 week Differential revision: https://reviews.freebsd.org/D42822 --- sys/compat/linuxkpi/common/include/linux/compat.h | 10 ++++++-- sys/compat/linuxkpi/common/include/linux/slab.h | 5 ++-- sys/compat/linuxkpi/common/src/linux_fpu.c | 28 ++++++++++++++++++++++- sys/compat/linuxkpi/common/src/linux_slab.c | 24 +++++++++++++++++++ 4 files changed, 62 insertions(+), 5 deletions(-) diff --git a/sys/compat/linuxkpi/common/include/linux/compat.h b/sys/compat/linuxkpi/common/include/linux/compat.h index d1a02f612f42..8a5a6918bb7c 100644 --- a/sys/compat/linuxkpi/common/include/linux/compat.h +++ b/sys/compat/linuxkpi/common/include/linux/compat.h @@ -41,17 +41,20 @@ extern int linux_alloc_current(struct thread *, int flags); extern void linux_free_current(struct task_struct *); extern struct domainset *linux_get_vm_domain_set(int node); +#define __current_unallocated(td) \ + __predict_false((td)->td_lkpi_task == NULL) + static inline void linux_set_current(struct thread *td) { - if (__predict_false(td->td_lkpi_task == NULL)) + if (__current_unallocated(td)) lkpi_alloc_current(td, M_WAITOK); } static inline int linux_set_current_flags(struct thread *td, int flags) { - if (__predict_false(td->td_lkpi_task == NULL)) + if (__current_unallocated(td)) return (lkpi_alloc_current(td, flags)); return (0); } @@ -59,4 +62,7 @@ linux_set_current_flags(struct thread *td, int flags) #define compat_ptr(x) ((void *)(uintptr_t)x) #define ptr_to_compat(x) ((uintptr_t)x) +typedef void fpu_safe_exec_cb_t(void *ctx); +void lkpi_fpu_safe_exec(fpu_safe_exec_cb_t func, void *ctx); + #endif /* _LINUXKPI_LINUX_COMPAT_H_ */ diff --git a/sys/compat/linuxkpi/common/include/linux/slab.h b/sys/compat/linuxkpi/common/include/linux/slab.h index 8557f831bb60..298306b6ea05 100644 --- a/sys/compat/linuxkpi/common/include/linux/slab.h +++ b/sys/compat/linuxkpi/common/include/linux/slab.h @@ -41,6 +41,7 @@ MALLOC_DECLARE(M_KMALLOC); +#define kmalloc(size, flags) lkpi_kmalloc(size, flags) #define kvmalloc(size, flags) kmalloc(size, flags) #define kvzalloc(size, flags) kmalloc(size, (flags) | __GFP_ZERO) #define kvcalloc(n, size, flags) kvmalloc_array(n, size, (flags) | __GFP_ZERO) @@ -53,7 +54,6 @@ MALLOC_DECLARE(M_KMALLOC); #define vmalloc_node(size, node) __vmalloc_node(size, GFP_KERNEL, node) #define vmalloc_user(size) __vmalloc(size, GFP_KERNEL | __GFP_ZERO, 0) #define vmalloc(size) __vmalloc(size, GFP_KERNEL, 0) -#define __kmalloc(...) kmalloc(__VA_ARGS__) /* * Prefix some functions with linux_ to avoid namespace conflict @@ -107,7 +107,7 @@ linux_check_m_flags(gfp_t flags) } static inline void * -kmalloc(size_t size, gfp_t flags) +__kmalloc(size_t size, gfp_t flags) { return (malloc(MAX(size, sizeof(struct llist_node)), M_KMALLOC, linux_check_m_flags(flags))); @@ -218,6 +218,7 @@ ksize(const void *ptr) return (malloc_usable_size(ptr)); } +extern void *lkpi_kmalloc(size_t size, gfp_t flags); extern struct linux_kmem_cache *linux_kmem_cache_create(const char *name, size_t size, size_t align, unsigned flags, linux_kmem_ctor_t *ctor); extern void *lkpi_kmem_cache_alloc(struct linux_kmem_cache *, gfp_t); diff --git a/sys/compat/linuxkpi/common/src/linux_fpu.c b/sys/compat/linuxkpi/common/src/linux_fpu.c index b26dce98774b..4e40a2b004bb 100644 --- a/sys/compat/linuxkpi/common/src/linux_fpu.c +++ b/sys/compat/linuxkpi/common/src/linux_fpu.c @@ -30,11 +30,13 @@ #include #include +#include #include #include -#if defined(__aarch64__) || defined(__amd64__) || defined(__i386__) +#if defined(__aarch64__) || defined(__arm__) || defined(__amd64__) || \ + defined(__i386__) || defined(__powerpc64__) #include @@ -58,6 +60,24 @@ lkpi_kernel_fpu_end(void) fpu_kern_leave(curthread, NULL); } +void +lkpi_fpu_safe_exec(fpu_safe_exec_cb_t func, void *ctx) +{ + unsigned int save_fpu_level; + + save_fpu_level = + __current_unallocated(curthread) ? 0 : current->fpu_ctx_level; + if (__predict_false(save_fpu_level != 0)) { + current->fpu_ctx_level = 1; + kernel_fpu_end(); + } + func(ctx); + if (__predict_false(save_fpu_level != 0)) { + kernel_fpu_begin(); + current->fpu_ctx_level = save_fpu_level; + } +} + #else void @@ -70,4 +90,10 @@ lkpi_kernel_fpu_end(void) { } +void +lkpi_fpu_safe_exec(fpu_safe_exec_cb_t func, void *ctx) +{ + func(ctx); +} + #endif diff --git a/sys/compat/linuxkpi/common/src/linux_slab.c b/sys/compat/linuxkpi/common/src/linux_slab.c index 72097c55f94c..68117d1c9fa7 100644 --- a/sys/compat/linuxkpi/common/src/linux_slab.c +++ b/sys/compat/linuxkpi/common/src/linux_slab.c @@ -25,6 +25,7 @@ */ #include +#include #include #include #include @@ -206,6 +207,29 @@ linux_kmem_cache_destroy(struct linux_kmem_cache *c) free(c, M_KMALLOC); } +struct lkpi_kmalloc_ctx { + size_t size; + gfp_t flags; + void *addr; +}; + +static void +lkpi_kmalloc_cb(void *ctx) +{ + struct lkpi_kmalloc_ctx *lmc = ctx; + + lmc->addr = __kmalloc(lmc->size, lmc->flags); +} + +void * +lkpi_kmalloc(size_t size, gfp_t flags) +{ + struct lkpi_kmalloc_ctx lmc = { .size = size, .flags = flags }; + + lkpi_fpu_safe_exec(&lkpi_kmalloc_cb, &lmc); + return(lmc.addr); +} + static void linux_kfree_async_fn(void *context, int pending) {