From nobody Mon Oct 21 15:05:15 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XXJWw1H4Vz5ZNrj; Mon, 21 Oct 2024 15:05:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XXJWv6Sy7z4fTk; Mon, 21 Oct 2024 15:05:15 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1729523115; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gYtwz7AD8FLK4icVLieLkRJ/qxPWOM9V9c21OSoXk+0=; b=HQ5/7Bd/7hjlhBszJHnHdQ7MA2ROEcfFqJf5ZYpKdVBI7jMc6TX1wPYDSX9xx9/CEO8bLZ ZBrhXq+1t4RPTeLc4ffIgSsJvPJer1oe+QHoZmSJcAulDMkvr5h7WsmumeJVHxDZCWFHf5 q3xePrVwFkNCFbnHprh7OU9eZU/+TTNhIzEfKAG2gE9tMJ4dWSSj9+V2oIviRMqcHtzWf1 cPoqc4LiXtCIvJjEdyEuGF4bj5fryBbXaCklmhJuRK2jP3KLJ+kkBErAPq6J/Flvby521q t3MpLJGLDctyMpayvWPWeaACQ/W6GF2oBFnw2rFphyu5ZraIiFDfAg1ougJefg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1729523115; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gYtwz7AD8FLK4icVLieLkRJ/qxPWOM9V9c21OSoXk+0=; b=i7D+bWYkIE02FK7E5QUzTyjBJYxofhxgF3ZU9OuJdw4pFkxOSpnS91pGsfJ+IOoRfIpHd3 FzQA2+qfpa59pTKzNCiGNniUvy5h0dPZkiBajaO5PJ2wDw8csM2gy/iTw5KvvDKrcOjr0T uyW9g3dFmg4tHW8vAHarPUPofRZGiRaZDQqNB3wWsm+8DI2AHmaZPxVLss2jWwHQoNex4Z aEwPIcjukG6eiPsuF42PuBYJe9r9bcyhfl5+nZBQj7NmorRY3gyQLXYh0Bdc4iEx5PZIzo hN7wMsn3k84GJ9G20KNGEHRYqGGRb16IhGmYkp2h0Xf8rD73aSkPOUw0QTMnhQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1729523115; a=rsa-sha256; cv=none; b=i8B3NIs3yPEhRViGJ6AM/gBbnU8nAlzu3Jl9eJ5oDjtrLr6RlIV8aL4NSbdjrhU/2+S3bl DiqC1b/h+wh/dgubQrGLM+cmaKVKzIjC7UYXMiHl3dzc1HlOJVj2aC+idLsgP6Kfs+6NgW h0TU7bVhURkQYuJLv70RvSOcmCQ8YS1WgVDxaHMEauLUQJT81Y/ZeWpqgmelbJLEch9wPP N6V4K1W6NASCZqnDdmes55RDs8VNTep6WL9Q5V4opZoxJ4TuD8i+QrVkXF+uksmAKNmJmG n2aHl3kUudlCbF2pELswIF7WgP961LnsBjIGP/1D8RQz7iP6Lz5UNrBkhcVT+g== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XXJWv5Cgpzb4w; Mon, 21 Oct 2024 15:05:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 49LF5FCj047204; Mon, 21 Oct 2024 15:05:15 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 49LF5Ffo047201; Mon, 21 Oct 2024 15:05:15 GMT (envelope-from git) Date: Mon, 21 Oct 2024 15:05:15 GMT Message-Id: <202410211505.49LF5Ffo047201@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: 1f9635e2c213 - stable/14 - arm64: Enable SVE in userspace List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 1f9635e2c213368cfed3737970f3814fbb55a922 Auto-Submitted: auto-generated The branch stable/14 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=1f9635e2c213368cfed3737970f3814fbb55a922 commit 1f9635e2c213368cfed3737970f3814fbb55a922 Author: Andrew Turner AuthorDate: 2024-09-27 14:06:03 +0000 Commit: Andrew Turner CommitDate: 2024-10-21 15:03:27 +0000 arm64: Enable SVE in userspace Report when SVE is present and allow it to be used by calling sve_restore_state on an SVE exception from userspace. Reviewed by: kib Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D43310 (cherry picked from commit 205c1007be6d57b2eb8620f79384cb2299dde40e) --- sys/arm64/arm64/identcpu.c | 82 ++++++++++++++++++++++++++++++++++++++-------- sys/arm64/arm64/trap.c | 6 ++-- sys/arm64/include/armreg.h | 1 + 3 files changed, 74 insertions(+), 15 deletions(-) diff --git a/sys/arm64/arm64/identcpu.c b/sys/arm64/arm64/identcpu.c index 84ad2d0eb282..22978af219dc 100644 --- a/sys/arm64/arm64/identcpu.c +++ b/sys/arm64/arm64/identcpu.c @@ -1384,13 +1384,10 @@ static const struct mrs_field_value id_aa64pfr0_sve[] = { MRS_FIELD_VALUE_END, }; -#if 0 -/* Enable when we add SVE support */ static const struct mrs_field_hwcap id_aa64pfr0_sve_caps[] = { MRS_HWCAP(1, HWCAP_SVE, ID_AA64PFR0_SVE_IMPL), MRS_HWCAP_END }; -#endif static const struct mrs_field_value id_aa64pfr0_ras[] = { MRS_FIELD_VALUE(ID_AA64PFR0_RAS_NONE, ""), @@ -1464,7 +1461,8 @@ static const struct mrs_field id_aa64pfr0_fields[] = { MRS_FIELD(ID_AA64PFR0, AMU, false, MRS_EXACT, id_aa64pfr0_amu), MRS_FIELD(ID_AA64PFR0, MPAM, false, MRS_EXACT, id_aa64pfr0_mpam), MRS_FIELD(ID_AA64PFR0, SEL2, false, MRS_EXACT, id_aa64pfr0_sel2), - MRS_FIELD(ID_AA64PFR0, SVE, false, MRS_EXACT, id_aa64pfr0_sve), + MRS_FIELD_HWCAP_SPLIT(ID_AA64PFR0, SVE, false, MRS_LOWER, MRS_EXACT, + id_aa64pfr0_sve, id_aa64pfr0_sve_caps), MRS_FIELD(ID_AA64PFR0, RAS, false, MRS_EXACT, id_aa64pfr0_ras), MRS_FIELD(ID_AA64PFR0, GIC, false, MRS_EXACT, id_aa64pfr0_gic), MRS_FIELD_HWCAP(ID_AA64PFR0, AdvSIMD, true, MRS_LOWER, @@ -1581,59 +1579,117 @@ static const struct mrs_field_value id_aa64zfr0_f64mm[] = { MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_f64mm_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEF64MM, ID_AA64ZFR0_F64MM_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_f32mm[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, F32MM, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_f32mm_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEF32MM, ID_AA64ZFR0_F32MM_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_i8mm[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, I8MM, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_i8mm_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEI8MM, ID_AA64ZFR0_I8MM_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_sm4[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, SM4, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_sm4_caps[] = { + MRS_HWCAP(2, HWCAP2_SVESM4, ID_AA64ZFR0_SM4_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_sha3[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, SHA3, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_sha3_caps[] = { + MRS_HWCAP(2, HWCAP2_SVESHA3, ID_AA64ZFR0_SHA3_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_bf16[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, BF16, NONE, BASE), MRS_FIELD_VALUE(ID_AA64ZFR0_BF16_EBF, "BF16+EBF"), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_bf16_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEBF16, ID_AA64ZFR0_BF16_BASE), + MRS_HWCAP(2, HWCAP2_SVE_EBF16, ID_AA64ZFR0_BF16_EBF), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_bitperm[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, BitPerm, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_bitperm_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEBITPERM, ID_AA64ZFR0_BitPerm_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_aes[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, AES, NONE, BASE), MRS_FIELD_VALUE(ID_AA64ZFR0_AES_PMULL, "AES+PMULL"), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_aes_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEAES, ID_AA64ZFR0_AES_BASE), + MRS_HWCAP(2, HWCAP2_SVEPMULL, ID_AA64ZFR0_AES_PMULL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_svever[] = { MRS_FIELD_VALUE(ID_AA64ZFR0_SVEver_SVE1, "SVE1"), MRS_FIELD_VALUE(ID_AA64ZFR0_SVEver_SVE2, "SVE2"), + MRS_FIELD_VALUE(ID_AA64ZFR0_SVEver_SVE2P1, "SVE2P1"), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_svever_caps[] = { + MRS_HWCAP(2, HWCAP2_SVE2, ID_AA64ZFR0_SVEver_SVE2), + MRS_HWCAP(2, HWCAP2_SVE2P1, ID_AA64ZFR0_SVEver_SVE2P1), + MRS_HWCAP_END, +}; + static const struct mrs_field id_aa64zfr0_fields[] = { - MRS_FIELD(ID_AA64ZFR0, F64MM, false, MRS_EXACT, id_aa64zfr0_f64mm), - MRS_FIELD(ID_AA64ZFR0, F32MM, false, MRS_EXACT, id_aa64zfr0_f32mm), - MRS_FIELD(ID_AA64ZFR0, I8MM, false, MRS_EXACT, id_aa64zfr0_i8mm), - MRS_FIELD(ID_AA64ZFR0, SM4, false, MRS_EXACT, id_aa64zfr0_sm4), - MRS_FIELD(ID_AA64ZFR0, SHA3, false, MRS_EXACT, id_aa64zfr0_sha3), - MRS_FIELD(ID_AA64ZFR0, BF16, false, MRS_EXACT, id_aa64zfr0_bf16), - MRS_FIELD(ID_AA64ZFR0, BitPerm, false, MRS_EXACT, id_aa64zfr0_bitperm), - MRS_FIELD(ID_AA64ZFR0, AES, false, MRS_EXACT, id_aa64zfr0_aes), - MRS_FIELD(ID_AA64ZFR0, SVEver, false, MRS_EXACT, id_aa64zfr0_svever), + MRS_FIELD_HWCAP(ID_AA64ZFR0, F64MM, false, MRS_LOWER, + id_aa64zfr0_f64mm, id_aa64zfr0_f64mm_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, F32MM, false, MRS_LOWER, + id_aa64zfr0_f32mm, id_aa64zfr0_f32mm_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, I8MM, false, MRS_LOWER, id_aa64zfr0_i8mm, + id_aa64zfr0_i8mm_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, SM4, false, MRS_LOWER, id_aa64zfr0_sm4, + id_aa64zfr0_sm4_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, SHA3, false, MRS_LOWER, id_aa64zfr0_sha3, + id_aa64zfr0_sha3_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, BF16, false, MRS_LOWER, id_aa64zfr0_bf16, + id_aa64zfr0_bf16_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, BitPerm, false, MRS_LOWER, + id_aa64zfr0_bitperm, id_aa64zfr0_bitperm_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, AES, false, MRS_LOWER, id_aa64zfr0_aes, + id_aa64zfr0_aes_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, SVEver, false, MRS_LOWER, + id_aa64zfr0_svever, id_aa64zfr0_svever_caps), MRS_FIELD_END, }; diff --git a/sys/arm64/arm64/trap.c b/sys/arm64/arm64/trap.c index 78582f5ac563..64a27b138a9a 100644 --- a/sys/arm64/arm64/trap.c +++ b/sys/arm64/arm64/trap.c @@ -635,8 +635,10 @@ do_el0_sync(struct thread *td, struct trapframe *frame) #endif break; case EXCP_SVE: - call_trapsignal(td, SIGILL, ILL_ILLTRP, (void *)frame->tf_elr, - exception); + /* Returns true if this thread can use SVE */ + if (!sve_restore_state(td)) + call_trapsignal(td, SIGILL, ILL_ILLTRP, + (void *)frame->tf_elr, exception); userret(td, frame); break; case EXCP_SVC32: diff --git a/sys/arm64/include/armreg.h b/sys/arm64/include/armreg.h index 2b92c036ef1c..6a3db5adc930 100644 --- a/sys/arm64/include/armreg.h +++ b/sys/arm64/include/armreg.h @@ -1535,6 +1535,7 @@ #define ID_AA64ZFR0_SVEver_VAL(x) ((x) & ID_AA64ZFR0_SVEver_MASK #define ID_AA64ZFR0_SVEver_SVE1 (UL(0x0) << ID_AA64ZFR0_SVEver_SHIFT) #define ID_AA64ZFR0_SVEver_SVE2 (UL(0x1) << ID_AA64ZFR0_SVEver_SHIFT) +#define ID_AA64ZFR0_SVEver_SVE2P1 (UL(0x2) << ID_AA64ZFR0_SVEver_SHIFT) #define ID_AA64ZFR0_AES_SHIFT 4 #define ID_AA64ZFR0_AES_MASK (UL(0xf) << ID_AA64ZFR0_AES_SHIFT) #define ID_AA64ZFR0_AES_VAL(x) ((x) & ID_AA64ZFR0_AES_MASK