From nobody Mon Dec 20 13:59:16 2021 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8E54D18FBF59; Mon, 20 Dec 2021 13:59:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JHh5x25nNz3DgF; Mon, 20 Dec 2021 13:59:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 19CB617981; Mon, 20 Dec 2021 13:59:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1BKDxGcS029300; Mon, 20 Dec 2021 13:59:16 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1BKDxG2U029299; Mon, 20 Dec 2021 13:59:16 GMT (envelope-from git) Date: Mon, 20 Dec 2021 13:59:16 GMT Message-Id: <202112201359.1BKDxG2U029299@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: c1381f07f61a - main - Don't sync the I/D caches when they are coherent List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: c1381f07f61a66979f1569995f37f2a0413c0413 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1640008757; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Dig7VOBeF05JqxjSR0wbdxcb4UWxbhW+usIU+3taWKg=; b=dxbTVqRfd77KMRiY5J63tNGARbRpVYxo0hWMR2BLOJ3naPCcEJkkqym+8OmJxMLYyZlWIQ EZ4cvR30D01iYbQKe0IUpk+PgmzTpS1cn2vFQIybyOmtF1NdAmLeSOz2IRrhpAamS86Ml/ RGkZY4u1jQrJfK8prJt2yME4WACnz9hzVUlGdKVevHZ76l9eTkUVFA2Pyn2gjUQd5Pl3ep 5hmOCv7n1HHbOxkTC6FBaE1QzblOhaIfl5vilkDJM7OuqZqNt/0eSWpatBhoMyD2FrwuZV AK2omBjD6iyhn+6eD6aKGnZ30nqiOyqJusBPphCzMy00hCVDDA5UFBR3tnzx3g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1640008757; a=rsa-sha256; cv=none; b=CgcoPJaqIkCb9PT/WFrKPYk94Hn+hCTKQEpD9ZCVKGZZThLmU0/aobs7B/j/FFDwZ1Cf8i wHXGkyW/sb36j23FS5B8CUnDmmXo6t0roQVABFZseuRquldI8R/AJEp2bLa1YxIZIFi2m0 8+xVSjd9y1lvlbymvAruObMZxMaaQ+oAf9KI35V2+8WB8CiEvKeEdlXnxD0dE1b7W47m5j P104ALakY1gcURWtjJBktYSImUBt4khkq4r2boNvz/u/ZtCq5AgLmxaiBLbvxdFBwj+YPH Oe/GQUEThAcb0FYmvEPqZuKMDjEtkRoIBHLYoFwxpnSh2QM/9lvD5gzF+k665w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=c1381f07f61a66979f1569995f37f2a0413c0413 commit c1381f07f61a66979f1569995f37f2a0413c0413 Author: Andrew Turner AuthorDate: 2021-12-17 09:33:57 +0000 Commit: Andrew Turner CommitDate: 2021-12-20 13:58:13 +0000 Don't sync the I/D caches when they are coherent In the arm64 loader we need to syncronise the I and D caches. On some newer CPUs the I and D caches are coherent so we don't need to perform these operations. While here remove the arguments to cpu_inval_icache as they are unneeded. Reported by: cperciva Tested by: cperciva Sponsored by: Innovate UK --- stand/arm64/libarm64/cache.c | 67 ++++++++++++++++++++++++-------------- stand/arm64/libarm64/cache.h | 2 +- stand/efi/loader/arch/arm64/exec.c | 2 +- 3 files changed, 45 insertions(+), 26 deletions(-) diff --git a/stand/arm64/libarm64/cache.c b/stand/arm64/libarm64/cache.c index 25766ef564dd..ff52572399ac 100644 --- a/stand/arm64/libarm64/cache.c +++ b/stand/arm64/libarm64/cache.c @@ -32,21 +32,30 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include "cache.h" +static bool +get_cache_dic(uint64_t ctr) +{ + return (CTR_DIC_VAL(ctr) != 0); +} + +static bool +get_cache_idc(uint64_t ctr) +{ + return (CTR_IDC_VAL(ctr) != 0); +} + static unsigned int -get_dcache_line_size(void) +get_dcache_line_size(uint64_t ctr) { - uint64_t ctr; unsigned int dcl_size; - /* Accessible from all security levels */ - ctr = READ_SPECIALREG(ctr_el0); - /* * Relevant field [19:16] is LOG2 * of the number of words in DCache line @@ -60,36 +69,46 @@ get_dcache_line_size(void) void cpu_flush_dcache(const void *ptr, size_t len) { - - uint64_t cl_size; + uint64_t cl_size, ctr; vm_offset_t addr, end; - cl_size = get_dcache_line_size(); - - /* Calculate end address to clean */ - end = (vm_offset_t)ptr + (vm_offset_t)len; - /* Align start address to cache line */ - addr = (vm_offset_t)ptr; - addr = rounddown2(addr, cl_size); + /* Accessible from all security levels */ + ctr = READ_SPECIALREG(ctr_el0); - for (; addr < end; addr += cl_size) - __asm __volatile("dc civac, %0" : : "r" (addr) : "memory"); - /* Full system DSB */ - __asm __volatile("dsb sy" : : : "memory"); + if (get_cache_idc(ctr)) { + dsb(ishst); + } else { + cl_size = get_dcache_line_size(ctr); + + /* Calculate end address to clean */ + end = (vm_offset_t)ptr + (vm_offset_t)len; + /* Align start address to cache line */ + addr = (vm_offset_t)ptr; + addr = rounddown2(addr, cl_size); + + for (; addr < end; addr += cl_size) + __asm __volatile("dc civac, %0" : : "r" (addr) : + "memory"); + /* Full system DSB */ + dsb(ish); + } } void -cpu_inval_icache(const void *ptr, size_t len) +cpu_inval_icache(void) { + uint64_t ctr; - /* NULL ptr or 0 len means all */ - if (ptr == NULL || len == 0) { + /* Accessible from all security levels */ + ctr = READ_SPECIALREG(ctr_el0); + + if (get_cache_dic(ctr)) { + isb(); + } else { __asm __volatile( "ic ialluis \n" "dsb ish \n" + "isb \n" : : : "memory"); - return; } - - /* TODO: Other cache ranges if necessary */ } diff --git a/stand/arm64/libarm64/cache.h b/stand/arm64/libarm64/cache.h index 89b094b19c18..5e560c4d578d 100644 --- a/stand/arm64/libarm64/cache.h +++ b/stand/arm64/libarm64/cache.h @@ -33,6 +33,6 @@ /* cache.c */ void cpu_flush_dcache(const void *, size_t); -void cpu_inval_icache(const void *, size_t); +void cpu_inval_icache(void); #endif /* _CACHE_H_ */ diff --git a/stand/efi/loader/arch/arm64/exec.c b/stand/efi/loader/arch/arm64/exec.c index 7783d46cd8e1..6cf4a4fd8e4d 100644 --- a/stand/efi/loader/arch/arm64/exec.c +++ b/stand/efi/loader/arch/arm64/exec.c @@ -128,7 +128,7 @@ elf64_exec(struct preloaded_file *fp) clean_size = (vm_offset_t)efi_translate(kernendp) - clean_addr; cpu_flush_dcache((void *)clean_addr, clean_size); - cpu_inval_icache(NULL, 0); + cpu_inval_icache(); (*entry)(modulep); panic("exec returned");