From nobody Tue Jan 04 10:50:18 2022 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2EFFE1936C6C; Tue, 4 Jan 2022 10:50:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JSqBy4rHWz4RJs; Tue, 4 Jan 2022 10:50:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 786A517851; Tue, 4 Jan 2022 10:50:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 204AoInY043730; Tue, 4 Jan 2022 10:50:18 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 204AoIDs043723; Tue, 4 Jan 2022 10:50:18 GMT (envelope-from git) Date: Tue, 4 Jan 2022 10:50:18 GMT Message-Id: <202201041050.204AoIDs043723@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: cc217b688132 - stable/13 - Don't sync the I/D caches when they are coherent List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: cc217b6881322596b2105fddae2c72d5a39ddd27 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1641293418; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9cwY8d8E1NZ7neNHyhvdbaZwtsb8mE15cwaXgtwCQxQ=; b=i8Xjf5UcWCfxczUpQ0tj6GtBf5GZLZ+PkYkJJUCozFLxs830ki9fbgCT/+aaj3HyDerwz0 K/l3jk4lVEJUbTarQnVlmwhtc6hC9NgfSoLV64I9FTAbxlqJM097S9M16ePPjHOlu7lhyh 4Cdh9tK9T83jLRVQr4C/r0pL6nbjjxTUdM/vdk5CVhVQnRMGIJvHxFQg4M58ssqC54zo7T EGOVGCu3lVSEk2l6tKqUBGy5UqFlsfoerZEIEmHNh9gNXAGoLPPuOUT+P+CTsCeDlc7rkX MxF5lXdWGBVIDhRP1BUcLv4LJWvX/mV35C+AvtKGyALxTpEoJf7A2sBnBU9/yg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1641293418; a=rsa-sha256; cv=none; b=JKYn9i9Xscdywu2R5yInyN5u6nN3rTYtwcziV4ytOZqN4ihENPAm/6eCkDzvF/1YA4QFKW FHIFdyuWXgP7l9Lu7GK2QHbNhH3rJlVCCHzEYnuAfv8qObkHLLmldIus7rA3XSpeaJ8vOA 5GMdD+PriqWf/KI371Qou5I2hsxmpDCeFsaVss7SItVUDSG224uTR11HmUKjgjKvsjoTyK +EYZVo9Ckuddh4mOafdKtCrJM3aem8A5ZTQz2QlDqEk1yravWF4VxyDEoemcNsSmUXVwD9 szyqUa57y5lp6Mdxd5+xY70cyVb3v4RJSoWHJcI1ZSq9kI1yosQ6tlRuaWXVzA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=cc217b6881322596b2105fddae2c72d5a39ddd27 commit cc217b6881322596b2105fddae2c72d5a39ddd27 Author: Andrew Turner AuthorDate: 2021-12-17 09:33:57 +0000 Commit: Andrew Turner CommitDate: 2022-01-04 10:08:31 +0000 Don't sync the I/D caches when they are coherent In the arm64 loader we need to syncronise the I and D caches. On some newer CPUs the I and D caches are coherent so we don't need to perform these operations. While here remove the arguments to cpu_inval_icache as they are unneeded. Reported by: cperciva Tested by: cperciva Sponsored by: Innovate UK (cherry picked from commit c1381f07f61a66979f1569995f37f2a0413c0413) --- stand/arm64/libarm64/cache.c | 67 ++++++++++++++++++++++++-------------- stand/arm64/libarm64/cache.h | 2 +- stand/efi/loader/arch/arm64/exec.c | 2 +- 3 files changed, 45 insertions(+), 26 deletions(-) diff --git a/stand/arm64/libarm64/cache.c b/stand/arm64/libarm64/cache.c index 25766ef564dd..ff52572399ac 100644 --- a/stand/arm64/libarm64/cache.c +++ b/stand/arm64/libarm64/cache.c @@ -32,21 +32,30 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include "cache.h" +static bool +get_cache_dic(uint64_t ctr) +{ + return (CTR_DIC_VAL(ctr) != 0); +} + +static bool +get_cache_idc(uint64_t ctr) +{ + return (CTR_IDC_VAL(ctr) != 0); +} + static unsigned int -get_dcache_line_size(void) +get_dcache_line_size(uint64_t ctr) { - uint64_t ctr; unsigned int dcl_size; - /* Accessible from all security levels */ - ctr = READ_SPECIALREG(ctr_el0); - /* * Relevant field [19:16] is LOG2 * of the number of words in DCache line @@ -60,36 +69,46 @@ get_dcache_line_size(void) void cpu_flush_dcache(const void *ptr, size_t len) { - - uint64_t cl_size; + uint64_t cl_size, ctr; vm_offset_t addr, end; - cl_size = get_dcache_line_size(); - - /* Calculate end address to clean */ - end = (vm_offset_t)ptr + (vm_offset_t)len; - /* Align start address to cache line */ - addr = (vm_offset_t)ptr; - addr = rounddown2(addr, cl_size); + /* Accessible from all security levels */ + ctr = READ_SPECIALREG(ctr_el0); - for (; addr < end; addr += cl_size) - __asm __volatile("dc civac, %0" : : "r" (addr) : "memory"); - /* Full system DSB */ - __asm __volatile("dsb sy" : : : "memory"); + if (get_cache_idc(ctr)) { + dsb(ishst); + } else { + cl_size = get_dcache_line_size(ctr); + + /* Calculate end address to clean */ + end = (vm_offset_t)ptr + (vm_offset_t)len; + /* Align start address to cache line */ + addr = (vm_offset_t)ptr; + addr = rounddown2(addr, cl_size); + + for (; addr < end; addr += cl_size) + __asm __volatile("dc civac, %0" : : "r" (addr) : + "memory"); + /* Full system DSB */ + dsb(ish); + } } void -cpu_inval_icache(const void *ptr, size_t len) +cpu_inval_icache(void) { + uint64_t ctr; - /* NULL ptr or 0 len means all */ - if (ptr == NULL || len == 0) { + /* Accessible from all security levels */ + ctr = READ_SPECIALREG(ctr_el0); + + if (get_cache_dic(ctr)) { + isb(); + } else { __asm __volatile( "ic ialluis \n" "dsb ish \n" + "isb \n" : : : "memory"); - return; } - - /* TODO: Other cache ranges if necessary */ } diff --git a/stand/arm64/libarm64/cache.h b/stand/arm64/libarm64/cache.h index 89b094b19c18..5e560c4d578d 100644 --- a/stand/arm64/libarm64/cache.h +++ b/stand/arm64/libarm64/cache.h @@ -33,6 +33,6 @@ /* cache.c */ void cpu_flush_dcache(const void *, size_t); -void cpu_inval_icache(const void *, size_t); +void cpu_inval_icache(void); #endif /* _CACHE_H_ */ diff --git a/stand/efi/loader/arch/arm64/exec.c b/stand/efi/loader/arch/arm64/exec.c index 45321261def0..a3021083dc08 100644 --- a/stand/efi/loader/arch/arm64/exec.c +++ b/stand/efi/loader/arch/arm64/exec.c @@ -127,7 +127,7 @@ elf64_exec(struct preloaded_file *fp) clean_size = (vm_offset_t)efi_translate(kernendp) - clean_addr; cpu_flush_dcache((void *)clean_addr, clean_size); - cpu_inval_icache(NULL, 0); + cpu_inval_icache(); (*entry)(modulep); panic("exec returned");