From nobody Thu Feb 09 10:01:23 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PCC7R5WxPz3nTHS; Thu, 9 Feb 2023 10:01:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PCC7R4Bxtz3CJ7; Thu, 9 Feb 2023 10:01:23 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675936883; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=h6ghMiel0uzKegbT8Se7rImi1/HGAD+jGo2d1kgXxAM=; b=WmHoYzymhGYASfVQ0Y+EcScWiKXVH6bWhxwOh0AUnMRfS4punZKV5l5o1flRWTYSofKO3d CeDvdqVxX+VDz7FFe0M85PTJVqX3/EqUMwhf0Nf93GuBkcQ75Bi6uw85fndAnVMyktmTm4 U+ZhUeXIi9BkPKSZqSNNv603A01QWeZCdeB5L17luTUgkManzHwO951ag328umerQ0h0Qn R/jdieyuwb82uALzXRk1Vsziz23ampEY91kHOcYEbP3dauo70ozY9GKiCrNtu/6BcWX5yV vM/AghzgHGo2XivMIEgjA49aG2cAuFfnS9OfM9KU1AT3bnEsMzENSG+TfWMfXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675936883; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=h6ghMiel0uzKegbT8Se7rImi1/HGAD+jGo2d1kgXxAM=; b=TomxYT2pWVjGDHRmbDzfHMrAAOdFW1Hmcu3oofdh+lG8yf5Tw9o/OCHLOdq2j5KHT4t3T/ vb7vxhC6udvBb5u3gjpGGOPGQ/kWcgLhk0ANhEi6C/fBGbkB7NqNoPKvZs2t15zV8EtJds 3+UTKS6pgYAEdQVSdfYFv+mc+c4Ax/DFdBBNzzRDF5at3SSIyc96FfDKnR6S/C2vz8IJfd KFq5hN74jziA580trw0qWZ+uImQGWrcemSJ0ehYYT+PJkY2YzK7OiCBAGTm56XC/Fe6k/x GgStb+JRcOYPjvLARCPi3FhcF9ttTNcpnxLDWC3vsPyJYIFUNyx796dtu+r19w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1675936883; a=rsa-sha256; cv=none; b=kwcAcC5hQlUeRiki6c1RLx0AsBOMwwBrnS+300d4o6Nlj8NRHsTRxbNUI/2hkMSzuIHyDK KsK2Lh28nhtepEKN8SDs/g62fQYJ7ZTLz52rAQe0NTxSyRBN47fySAmwvUQnz9JjiQFhMs VyiTJ3k8V7LHGhqiRVoX04zced+cEhSOQ+CaC8jUgtvx2OoOb9LRNghFOx+rhpSVu+dCMg B3A+UcO2pTXKG3ufiVenPTaAXmLUYI3Wmy6YrVxYXMuG0Bv35qXGStHVhlM8S28zSFmx/0 BQdGI5MSTaP87lLioI27l/KubMbDArZvaL4MQ1wEvzYLd8dRPojt8BHAH+8FgQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PCC7R1q5tzY80; Thu, 9 Feb 2023 10:01:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 319A1N0D077922; Thu, 9 Feb 2023 10:01:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 319A1NJU077921; Thu, 9 Feb 2023 10:01:23 GMT (envelope-from git) Date: Thu, 9 Feb 2023 10:01:23 GMT Message-Id: <202302091001.319A1NJU077921@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: da57cc1400a0 - stable/13 - Read the arm64 far early in el0 exceptions List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: da57cc1400a0bff21ff2dc239404a4a1d2af7539 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=da57cc1400a0bff21ff2dc239404a4a1d2af7539 commit da57cc1400a0bff21ff2dc239404a4a1d2af7539 Author: Andrew Turner AuthorDate: 2023-01-25 17:47:39 +0000 Commit: Andrew Turner CommitDate: 2023-02-09 09:53:56 +0000 Read the arm64 far early in el0 exceptions When handling userspace exceptions on arm64 we need to dereference the current thread pointer. If this is being promoted/demoted there is a small window where it will cause another exception to be hit. As this second exception will set the fault address register we will read the incorrect value in the userspace exception handler. Fix this be always reading the fault address before dereferencing the current thread pointer. Reported by: olivier@ Reviewed by: markj Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D38196 (cherry picked from commit f29942229d24ebb8b98f8c5d02f3c8632648007e) --- sys/arm64/arm64/exception.S | 15 +++++++++++++++ sys/arm64/arm64/trap.c | 26 +++++++------------------- 2 files changed, 22 insertions(+), 19 deletions(-) diff --git a/sys/arm64/arm64/exception.S b/sys/arm64/arm64/exception.S index 63fcbd2d98a7..1ee784a48e40 100644 --- a/sys/arm64/arm64/exception.S +++ b/sys/arm64/arm64/exception.S @@ -210,10 +210,25 @@ ENTRY(handle_el1h_irq) END(handle_el1h_irq) ENTRY(handle_el0_sync) + /* + * Read the fault address early. The current thread structure may + * be transiently unmapped if it is part of a memory range being + * promoted or demoted to/from a superpage. As this involves a + * break-before-make sequence there is a short period of time where + * an access will raise an exception. If this happens the fault + * address will be changed to the kernel address so a later read of + * far_el1 will give the wrong value. + * + * The earliest memory access that could trigger a fault is in a + * function called by the save_registers macro so this is the latest + * we can read the userspace value. + */ + mrs x19, far_el1 save_registers 0 ldr x0, [x18, #PC_CURTHREAD] mov x1, sp str x1, [x0, #TD_FRAME] + mov x2, x19 bl do_el0_sync do_ast restore_registers 0 diff --git a/sys/arm64/arm64/trap.c b/sys/arm64/arm64/trap.c index e845f998633a..7d36a9c9dee1 100644 --- a/sys/arm64/arm64/trap.c +++ b/sys/arm64/arm64/trap.c @@ -76,7 +76,7 @@ __FBSDID("$FreeBSD$"); /* Called from exception.S */ void do_el1h_sync(struct thread *, struct trapframe *); -void do_el0_sync(struct thread *, struct trapframe *); +void do_el0_sync(struct thread *, struct trapframe *, uint64_t far); void do_el0_error(struct trapframe *); void do_serror(struct trapframe *); void unhandled_exception(struct trapframe *); @@ -527,11 +527,11 @@ do_el1h_sync(struct thread *td, struct trapframe *frame) } void -do_el0_sync(struct thread *td, struct trapframe *frame) +do_el0_sync(struct thread *td, struct trapframe *frame, uint64_t far) { pcpu_bp_harden bp_harden; uint32_t exception; - uint64_t esr, far; + uint64_t esr; int dfsc; /* Check we have a sane environment when entering from userland */ @@ -541,27 +541,15 @@ do_el0_sync(struct thread *td, struct trapframe *frame) esr = frame->tf_esr; exception = ESR_ELx_EXCEPTION(esr); - switch (exception) { - case EXCP_INSN_ABORT_L: - far = READ_SPECIALREG(far_el1); - + if (exception == EXCP_INSN_ABORT_L && far > VM_MAXUSER_ADDRESS) { /* * Userspace may be trying to train the branch predictor to * attack the kernel. If we are on a CPU affected by this * call the handler to clear the branch predictor state. */ - if (far > VM_MAXUSER_ADDRESS) { - bp_harden = PCPU_GET(bp_harden); - if (bp_harden != NULL) - bp_harden(); - } - break; - case EXCP_UNKNOWN: - case EXCP_DATA_ABORT_L: - case EXCP_DATA_ABORT: - case EXCP_WATCHPT_EL0: - far = READ_SPECIALREG(far_el1); - break; + bp_harden = PCPU_GET(bp_harden); + if (bp_harden != NULL) + bp_harden(); } intr_enable();