From nobody Thu Sep 29 14:12:58 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Mdb1K39jGz4cx1R; Thu, 29 Sep 2022 14:13:09 +0000 (UTC) (envelope-from andrew@fubar.geek.nz) Received: from fry.fubar.geek.nz (fry.fubar.geek.nz [139.59.165.16]) by mx1.freebsd.org (Postfix) with ESMTP id 4Mdb1H1lcgz3lvx; Thu, 29 Sep 2022 14:13:06 +0000 (UTC) (envelope-from andrew@fubar.geek.nz) Received: from smtpclient.apple (cpc91214-cmbg18-2-0-cust234.5-4.cable.virginm.net [81.102.75.235]) by fry.fubar.geek.nz (Postfix) with ESMTPSA id 75C6E4E703; Thu, 29 Sep 2022 14:12:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fubar.geek.nz; s=mail; t=1664460779; bh=1PSxgMemQQuJBOwrztZS1ReEVGh7j6dnoKcQEVYOdRc=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=imKFlorQl31++Lb32A/gzMFvrriqEnBgnPTnglCOQnXJKJiVZJVqJ57frob45i9Xe tIMgzQuLdI/h6DcWMD5KAaxjkkuoQD5br/Waiutg0cP4u4nhDuTrRAaPEw4ZiDmVfj sYz8v4qoqZ0BklWS+dsWaH4KAm99nom6VlT/pzSSGrNCGhzin+Hco6g573ozp9IjET FPibni0j3VbSLshaEYsvyXBZ07BtgvnK4n9lyrt+WvZDpWCPoKzAY+AxvOPdNNrp3m 9QdcyYnjMZDc3y62lgTvCi+dkw8F69sW24GQBZZ8uBKMJSdYQI0nds+OHZ4docFE1S UdnTVUoN5h+jzVQCcA/AyZfX1AiJ2vdQHLUmXZo4x8fq9hXVe6i1ncZDtEAxTgMJet Y+DcuOvRmuYLDWXS1/xIFSGg5okbBovmTMfxdqpZ4zvNkAmaNA+vfYTTPZPtpyARWS 3WgemyQpOmxHN/s0FCGCFID9/aNyhJ2FOUDbT050bibo0GagwuLifV9SUl+UjAXfWJ RHXuEhhaKwlWGSd6sFL6O92QahmU4Mzvmu3ivAJOoYyx/U+qgpKAHTwXdU4ttTM7xH K/j6iph53zOnyKC6ovw6v25Cf5nOcWq6j3G6Ne+Jbe5IafoAJp99C9+72HONV0GkGM XGE3TCC4pA+0mtaicpK3isMY= Content-Type: text/plain; charset=us-ascii List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: git: 6cf00ef80c20 - main - arm64: enablement for ARM64 in Hyper-V (Part 1) From: Andrew Turner In-Reply-To: <202209291019.28TAJoks018126@gitrepo.freebsd.org> Date: Thu, 29 Sep 2022 15:12:58 +0100 Cc: "src-committers@freebsd.org" , "dev-commits-src-all@freebsd.org" , "dev-commits-src-main@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <02EE030E-41E4-46A9-AD75-CA9612770275@fubar.geek.nz> References: <202209291019.28TAJoks018126@gitrepo.freebsd.org> To: Wei Hu X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4Mdb1H1lcgz3lvx X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fubar.geek.nz header.s=mail header.b=imKFlorQ; dmarc=pass (policy=none) header.from=fubar.geek.nz; spf=softfail (mx1.freebsd.org: 139.59.165.16 is neither permitted nor denied by domain of andrew@fubar.geek.nz) smtp.mailfrom=andrew@fubar.geek.nz X-Spamd-Result: default: False [0.31 / 15.00]; VIOLATED_DIRECT_SPF(3.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.995]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50)[]; R_DKIM_ALLOW(-0.20)[fubar.geek.nz:s=mail]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; FREEFALL_USER(0.00)[andrew]; FROM_HAS_DN(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; ARC_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:139.59.160.0/20, country:US]; MLMMJ_DEST(0.00)[dev-commits-src-all@FreeBSD.org,dev-commits-src-main@FreeBSD.org]; R_SPF_SOFTFAIL(0.00)[~all]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[fubar.geek.nz:+]; FROM_EQ_ENVFROM(0.00)[]; DMARC_POLICY_ALLOW(0.00)[fubar.geek.nz,none]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N It looks like you have the contents of = sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c duplicated. Andrew > On 29 Sep 2022, at 11:19, Wei Hu wrote: >=20 > The branch main has been updated by whu: >=20 > URL: = https://cgit.FreeBSD.org/src/commit/?id=3D6cf00ef80c2014811bb6ce35e969b95d= fe574793 >=20 > commit 6cf00ef80c2014811bb6ce35e969b95dfe574793 > Author: Souradeep Chakrabarti > AuthorDate: 2022-09-29 09:57:46 +0000 > Commit: Wei Hu > CommitDate: 2022-09-29 10:14:41 +0000 >=20 > arm64: enablement for ARM64 in Hyper-V (Part 1) >=20 > The changes are to refactor the code of vmbus.c and hyperv.c to = keep minimal > arch specific codes there and have them in separate files in x86/ = arm64/ . > x86 is a new directory, which contains codes for x86 / x86_64. = Instead of > repeating the same codes in existing amd64/ and i386/, this = approach reduced > the repetition. This is first of three patches for Hyper-V = enablement. >=20 > Reviewed by: whu > Tested by: Souradeep Chakrabarti > Sponsored by: Microsoft > Differential Revision: https://reviews.freebsd.org/D36466 > --- > sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c | 190 +++++++++++++ > sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.c | 103 +++++++ > sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.h | 59 ++++ > sys/dev/hyperv/vmbus/aarch64/hyperv_reg.h | 386 = ++++++++++++++++++++++++++ > sys/dev/hyperv/vmbus/aarch64/vmbus_aarch64.c | 314 = +++++++++++++++++++++ > 5 files changed, 1052 insertions(+) >=20 > diff --git a/sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c = b/sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c > new file mode 100644 > index 000000000000..6104e88130db > --- /dev/null > +++ b/sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c > @@ -0,0 +1,190 @@ > +/*- SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * Copyright (c) 2009-2012,2016-2017, 2022 Microsoft Corp. > + * Copyright (c) 2012 NetApp Inc. > + * Copyright (c) 2012 Citrix Inc. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice unmodified, this list of conditions, and the following > + * disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS = OR > + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED = WARRANTIES > + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE = DISCLAIMED. > + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, > + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES = (INCLUDING, BUT > + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS = OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON = ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR = TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE = USE OF > + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +/** > + * Implements low-level interactions with Hyper-V/Azure > + */ > + > +#include > +__FBSDID("$FreeBSD$"); > + > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > + > +void hyperv_init_tc(void); > +int hypercall_page_setup(vm_paddr_t); > +void hypercall_disable(void); > +bool hyperv_identify_features(void); > + > +u_int hyperv_ver_major; > +u_int hyperv_features; > +u_int hyperv_recommends; > + > +hyperv_tc64_t hyperv_tc64; > + > +void > +hyperv_init_tc(void) > +{ > + hyperv_tc64 =3D NULL; > +} > + > +int > +hypercall_page_setup(vm_paddr_t hc) > +{ > + return (0); > +} > + > +void > +hypercall_disable(void) > +{ > + return; > +} > + > +bool > +hyperv_identify_features(void) > +{ > + struct hv_get_vp_registers_output result; > + vm_guest =3D VM_GUEST_HV; > + > + hv_get_vpreg_128(CPUID_LEAF_HV_FEATURES, &result); > + hyperv_features =3D result.as32.a; > + hv_get_vpreg_128(CPUID_LEAF_HV_IDENTITY, &result); > + hyperv_ver_major =3D result.as32.b >> 16; > + hv_get_vpreg_128(CPUID_LEAF_HV_RECOMMENDS, &result); > + hyperv_recommends =3D result.as32.a; > + return (true); > +} > +/*- SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * Copyright (c) 2009-2012,2016-2017, 2022 Microsoft Corp. > + * Copyright (c) 2012 NetApp Inc. > + * Copyright (c) 2012 Citrix Inc. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice unmodified, this list of conditions, and the following > + * disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS = OR > + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED = WARRANTIES > + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE = DISCLAIMED. > + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, > + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES = (INCLUDING, BUT > + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS = OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON = ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR = TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE = USE OF > + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +/** > + * Implements low-level interactions with Hyper-V/Azure > + */ > + > +#include > +__FBSDID("$FreeBSD$"); > + > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > + > +void hyperv_init_tc(void); > +int hypercall_page_setup(vm_paddr_t); > +void hypercall_disable(void); > +bool hyperv_identify_features(void); > + > +u_int hyperv_ver_major; > +u_int hyperv_features; > +u_int hyperv_recommends; > + > +hyperv_tc64_t hyperv_tc64; > + > +void > +hyperv_init_tc(void) > +{ > + hyperv_tc64 =3D NULL; > +} > + > +int > +hypercall_page_setup(vm_paddr_t hc) > +{ > + return (0); > +} > + > +void > +hypercall_disable(void) > +{ > + return; > +} > + > +bool > +hyperv_identify_features(void) > +{ > + struct hv_get_vp_registers_output result; > + vm_guest =3D VM_GUEST_HV; > + > + hv_get_vpreg_128(CPUID_LEAF_HV_FEATURES, &result); > + hyperv_features =3D result.as32.a; > + hv_get_vpreg_128(CPUID_LEAF_HV_IDENTITY, &result); > + hyperv_ver_major =3D result.as32.b >> 16; > + hv_get_vpreg_128(CPUID_LEAF_HV_RECOMMENDS, &result); > + hyperv_recommends =3D result.as32.a; > + return (true); > +} > diff --git a/sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.c = b/sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.c > new file mode 100644 > index 000000000000..9bfe34044335 > --- /dev/null > +++ b/sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.c > @@ -0,0 +1,103 @@ > +/*- SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * > + * Copyright (c) 2016-2017,2022 Microsoft Corp. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice unmodified, this list of conditions, and the following > + * disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS = OR > + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED = WARRANTIES > + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE = DISCLAIMED. > + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, > + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES = (INCLUDING, BUT > + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS = OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON = ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR = TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE = USE OF > + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +#include > +__FBSDID("$FreeBSD$"); > + > +#include > +#include > +#include > +#include > +#include > +#include > + > +#define HVCALL_SET_VP_REGISTERS 0x0051 > +#define HVCALL_GET_VP_REGISTERS 0x0050 > +#define BIT(A) (1 << (A)) > +#define HV_HYPERCALL_FAST_BIT BIT(16) > +#define BIT_ULL(a) (1ULL << (a)) > +#define HV_HYPERCALL_REP_COMP_1 BIT_ULL(32) > +#define HV_PARTITION_ID_SELF ((u64)-1) > +#define HV_VP_INDEX_SELF ((u32)-2) > +#define HV_SMCCC_FUNC_NUMBER 1 > + > +#define HV_FUNC_ID \ > + SMCCC_FUNC_ID(SMCCC_YIELDING_CALL, SMCCC_64BIT_CALL, \ > + SMCCC_VENDOR_HYP_SERVICE_CALLS, HV_SMCCC_FUNC_NUMBER) > + > +void > +arm_hv_set_vreg(u32 msr, u64 value) > +{ > + arm_smccc_hvc(HV_FUNC_ID, > + HVCALL_SET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT | > + HV_HYPERCALL_REP_COMP_1, > + HV_PARTITION_ID_SELF, HV_VP_INDEX_SELF, msr, 0, value, 0, = NULL); > +} > + > +void > +hv_get_vpreg_128(u32 msr, struct hv_get_vp_registers_output *result) > +{ > + struct arm_smccc_1_2_regs args; > + struct arm_smccc_1_2_regs res; > + > + args.a0 =3D HV_FUNC_ID; > + args.a1 =3D HVCALL_GET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT | > + HV_HYPERCALL_REP_COMP_1; > + args.a2 =3D HV_PARTITION_ID_SELF; > + args.a3 =3D HV_VP_INDEX_SELF; > + args.a4 =3D msr; > + > + /* > + * Use the SMCCC 1.2 interface because the results are in = registers > + * beyond X0-X3. > + */ > + arm_smccc_1_2_hvc(&args, &res); > + > + result->as64.low =3D res.a6; > + result->as64.high =3D res.a7; > +} > + > +u64 > +arm_hv_get_vreg(u32 msr) > +{ > + struct hv_get_vp_registers_output output; > + > + hv_get_vpreg_128(msr, &output); > + > + return (output.as64.low); > +} > + > +uint64_t > +hypercall_md(volatile void *hc_addr, uint64_t in_val, uint64_t = in_paddr, > + uint64_t out_paddr) > +{ > + struct arm_smccc_res res; > + > + arm_smccc_hvc(HV_FUNC_ID, in_val, in_paddr, out_paddr, 0, 0, 0, = 0, > + &res); > + > + return (res.a0); > +} > diff --git a/sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.h = b/sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.h > new file mode 100644 > index 000000000000..b017fdbc14c9 > --- /dev/null > +++ b/sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.h > @@ -0,0 +1,59 @@ > +/*- SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * > + * Copyright (c) 2022 Microsoft Corp. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice unmodified, this list of conditions, and the following > + * disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS = OR > + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED = WARRANTIES > + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE = DISCLAIMED. > + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, > + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES = (INCLUDING, BUT > + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS = OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON = ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR = TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE = USE OF > + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + * > + * $FreeBSD$ > + */ > + > +#ifndef _HYPERV_MACHDEP_H_ > +#define _HYPERV_MACHDEP_H_ > + > +#include > + > +typedef uint32_t u32; > +typedef uint64_t u64; > +struct hv_get_vp_registers_output { > + union { > + struct { > + u32 a; > + u32 b; > + u32 c; > + u32 d; > + } as32 __packed; > + struct { > + u64 low; > + u64 high; > + } as64 __packed; > + }; > +}; > + > +uint64_t hypercall_md(volatile void *hc_addr, u64 in_val, > + u64 in_paddr, u64 out_paddr); > +void hv_get_vpreg_128(u32, struct hv_get_vp_registers_output *); > +void arm_hv_set_vreg(u32 msr, u64 val); > +#define WRMSR(msr, val) arm_hv_set_vreg(msr, val) > +u64 arm_hv_get_vreg(u32 msr); > +#define RDMSR(msr) arm_hv_get_vreg(msr) > + > +#endif /* !_HYPERV_MACHDEP_H_ */ > diff --git a/sys/dev/hyperv/vmbus/aarch64/hyperv_reg.h = b/sys/dev/hyperv/vmbus/aarch64/hyperv_reg.h > new file mode 100644 > index 000000000000..132b40768790 > --- /dev/null > +++ b/sys/dev/hyperv/vmbus/aarch64/hyperv_reg.h > @@ -0,0 +1,386 @@ > +/*- SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * Copyright (c) 2022 Microsoft Corp. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice unmodified, this list of conditions, and the following > + * disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS = OR > + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED = WARRANTIES > + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE = DISCLAIMED. > + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, > + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES = (INCLUDING, BUT > + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS = OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON = ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR = TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE = USE OF > + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + * > + * $FreeBSD$ > + */ > + > +#ifndef _HYPERV_REG_H_ > +#define _HYPERV_REG_H_ > + > +#include > +#include > + > +/* > + * Hyper-V Synthetic MSRs > + */ > + > +#define MSR_HV_GUEST_OS_ID 0x00090002 > +#define MSR_HV_GUESTID_BUILD_MASK 0xffffULL > +#define MSR_HV_GUESTID_VERSION_MASK 0x0000ffffffff0000ULL > +#define MSR_HV_GUESTID_VERSION_SHIFT 16 > +#define MSR_HV_GUESTID_OSID_MASK 0x00ff000000000000ULL > +#define MSR_HV_GUESTID_OSID_SHIFT 48 > +#define MSR_HV_GUESTID_OSTYPE_MASK 0x7f00000000000000ULL > +#define MSR_HV_GUESTID_OSTYPE_SHIFT 56 > +#define MSR_HV_GUESTID_OPENSRC 0x8000000000000000ULL > +#define MSR_HV_GUESTID_OSTYPE_LINUX \ > + ((0x01ULL << MSR_HV_GUESTID_OSTYPE_SHIFT) | = MSR_HV_GUESTID_OPENSRC) > +#define MSR_HV_GUESTID_OSTYPE_FREEBSD \ > + ((0x02ULL << MSR_HV_GUESTID_OSTYPE_SHIFT) | = MSR_HV_GUESTID_OPENSRC) > + > +#define MSR_HV_HYPERCALL 0x40000001 > +#define MSR_HV_HYPERCALL_ENABLE 0x0001ULL > +#define MSR_HV_HYPERCALL_RSVD_MASK 0x0ffeULL > +#define MSR_HV_HYPERCALL_PGSHIFT 12 > + > +#define MSR_HV_VP_INDEX 0x00090003 > + > +#define MSR_HV_REFERENCE_TSC 0x40000021 > +#define MSR_HV_REFTSC_ENABLE 0x0001ULL > +#define MSR_HV_REFTSC_RSVD_MASK 0x0ffeULL > +#define MSR_HV_REFTSC_PGSHIFT 12 > + > +#define MSR_HV_SCONTROL 0x000A0010 > +#define MSR_HV_SCTRL_ENABLE 0x0001ULL > +#define MSR_HV_SCTRL_RSVD_MASK 0xfffffffffffffffeULL > + > +#define MSR_HV_SIEFP 0x000A0012 > +#define MSR_HV_SIEFP_ENABLE 0x0001ULL > +#define MSR_HV_SIEFP_RSVD_MASK 0x0ffeULL > +#define MSR_HV_SIEFP_PGSHIFT 12 > + > +#define MSR_HV_SIMP 0x000A0013 > +#define MSR_HV_SIMP_ENABLE 0x0001ULL > +#define MSR_HV_SIMP_RSVD_MASK 0x0ffeULL > +#define MSR_HV_SIMP_PGSHIFT 12 > + > +#define MSR_HV_EOM 0x000A0014 > + > +#define MSR_HV_SINT0 0x000A0000 > +#define MSR_HV_SINT_VECTOR_MASK 0x00ffULL > +#define MSR_HV_SINT_RSVD1_MASK 0xff00ULL > +#define MSR_HV_SINT_MASKED 0x00010000ULL > +#define MSR_HV_SINT_AUTOEOI 0x00000000ULL > +#define MSR_HV_SINT_RSVD2_MASK 0xfffffffffffc0000ULL > +#define MSR_HV_SINT_RSVD_MASK (MSR_HV_SINT_RSVD1_MASK = | \ > + MSR_HV_SINT_RSVD2_MASK) > + > +#define MSR_HV_STIMER0_CONFIG 0x400000b0 > +#define MSR_HV_STIMER_CFG_ENABLE 0x0001ULL > +#define MSR_HV_STIMER_CFG_PERIODIC 0x0002ULL > +#define MSR_HV_STIMER_CFG_LAZY 0x0004ULL > +#define MSR_HV_STIMER_CFG_AUTOEN 0x0008ULL > +#define MSR_HV_STIMER_CFG_SINT_MASK 0x000f0000ULL > +#define MSR_HV_STIMER_CFG_SINT_SHIFT 16 > + > +#define MSR_HV_STIMER0_COUNT 0x400000b1 > + > +/* > + * CPUID leaves > + */ > + > +#define CPUID_LEAF_HV_MAXLEAF 0x40000000 > + > +#define CPUID_LEAF_HV_INTERFACE 0x40000001 > +#define CPUID_HV_IFACE_HYPERV 0x31237648 /* HV#1 = */ > + > +#define CPUID_LEAF_HV_IDENTITY 0x00000100 > + > +#define CPUID_LEAF_HV_FEATURES 0x00000200 > +/* EAX: features include/hyperv.h CPUID_HV_MSR */ > +/* ECX: power management features */ > +#define CPUPM_HV_CSTATE_MASK 0x000f /* deepest C-state */ > +#define CPUPM_HV_C3_HPET 0x0010 /* C3 requires HPET */ > +#define CPUPM_HV_CSTATE(f) ((f) & CPUPM_HV_CSTATE_MASK) > +/* EDX: features3 */ > +#define CPUID3_HV_MWAIT 0x0001 /* MWAIT */ > +#define CPUID3_HV_XMM_HYPERCALL 0x0010 /* Hypercall = input through > + * XMM regs */ > +#define CPUID3_HV_GUEST_IDLE 0x0020 /* guest idle */ > +#define CPUID3_HV_NUMA 0x0080 /* NUMA distance = query */ > +#define CPUID3_HV_TIME_FREQ 0x0100 /* timer frequency query > + * (TSC, LAPIC) */ > +#define CPUID3_HV_MSR_CRASH 0x0400 /* MSRs for guest crash = */ > + > +#define CPUID_LEAF_HV_RECOMMENDS 0x00000201 > +#define CPUID_LEAF_HV_LIMITS 0x40000005 > +#define CPUID_LEAF_HV_HWFEATURES 0x40000006 > + > +/* > + * Hyper-V Monitor Notification Facility > + */ > +struct hyperv_mon_param { > + uint32_t mp_connid; > + uint16_t mp_evtflag_ofs; > + uint16_t mp_rsvd; > +} __packed; > + > +/* > + * Hyper-V message types > + */ > +#define HYPERV_MSGTYPE_NONE 0 > +#define HYPERV_MSGTYPE_CHANNEL 1 > +#define HYPERV_MSGTYPE_TIMER_EXPIRED 0x80000010 > + > +/* > + * Hypercall status codes > + */ > +#define HYPERCALL_STATUS_SUCCESS 0x0000 > + > +/* > + * Hypercall input values > + */ > +#define HYPERCALL_POST_MESSAGE 0x005c > +#define HYPERCALL_SIGNAL_EVENT 0x005d > + > +/* > + * Hypercall input parameters > + */ > +#define HYPERCALL_PARAM_ALIGN 8 > +#if 0 > +/* > + * XXX > + * <> requires > + * input parameters size to be multiple of 8, however, many post > + * message input parameters do _not_ meet this requirement. > + */ > +#define HYPERCALL_PARAM_SIZE_ALIGN 8 > +#endif > + > +/* > + * HYPERCALL_POST_MESSAGE > + */ > +#define HYPERCALL_POSTMSGIN_DSIZE_MAX 240 > +#define HYPERCALL_POSTMSGIN_SIZE 256 > + > +struct hypercall_postmsg_in { > + uint32_t hc_connid; > + uint32_t hc_rsvd; > + uint32_t hc_msgtype; /* HYPERV_MSGTYPE_ */ > + uint32_t hc_dsize; > + uint8_t hc_data[HYPERCALL_POSTMSGIN_DSIZE_MAX]; > +} __packed; > +CTASSERT(sizeof(struct hypercall_postmsg_in) =3D=3D = HYPERCALL_POSTMSGIN_SIZE); > + > +/* > + * HYPERCALL_SIGNAL_EVENT > + * > + * struct hyperv_mon_param. > + */ > + > +#endif /* !_HYPERV_REG_H_ */ > +/*- SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * Copyright (c) 2022 Microsoft Corp. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice unmodified, this list of conditions, and the following > + * disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS = OR > + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED = WARRANTIES > + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE = DISCLAIMED. > + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, > + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES = (INCLUDING, BUT > + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS = OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON = ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR = TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE = USE OF > + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + * > + * $FreeBSD$ > + */ > + > +#ifndef _HYPERV_REG_H_ > +#define _HYPERV_REG_H_ > + > +#include > +#include > + > +/* > + * Hyper-V Synthetic MSRs > + */ > + > +#define MSR_HV_GUEST_OS_ID 0x00090002 > +#define MSR_HV_GUESTID_BUILD_MASK 0xffffULL > +#define MSR_HV_GUESTID_VERSION_MASK 0x0000ffffffff0000ULL > +#define MSR_HV_GUESTID_VERSION_SHIFT 16 > +#define MSR_HV_GUESTID_OSID_MASK 0x00ff000000000000ULL > +#define MSR_HV_GUESTID_OSID_SHIFT 48 > +#define MSR_HV_GUESTID_OSTYPE_MASK 0x7f00000000000000ULL > +#define MSR_HV_GUESTID_OSTYPE_SHIFT 56 > +#define MSR_HV_GUESTID_OPENSRC 0x8000000000000000ULL > +#define MSR_HV_GUESTID_OSTYPE_LINUX \ > + ((0x01ULL << MSR_HV_GUESTID_OSTYPE_SHIFT) | = MSR_HV_GUESTID_OPENSRC) > +#define MSR_HV_GUESTID_OSTYPE_FREEBSD \ > + ((0x02ULL << MSR_HV_GUESTID_OSTYPE_SHIFT) | = MSR_HV_GUESTID_OPENSRC) > + > +#define MSR_HV_HYPERCALL 0x40000001 > +#define MSR_HV_HYPERCALL_ENABLE 0x0001ULL > +#define MSR_HV_HYPERCALL_RSVD_MASK 0x0ffeULL > +#define MSR_HV_HYPERCALL_PGSHIFT 12 > + > +#define MSR_HV_VP_INDEX 0x00090003 > + > +#define MSR_HV_REFERENCE_TSC 0x40000021 > +#define MSR_HV_REFTSC_ENABLE 0x0001ULL > +#define MSR_HV_REFTSC_RSVD_MASK 0x0ffeULL > +#define MSR_HV_REFTSC_PGSHIFT 12 > + > +#define MSR_HV_SCONTROL 0x000A0010 > +#define MSR_HV_SCTRL_ENABLE 0x0001ULL > +#define MSR_HV_SCTRL_RSVD_MASK 0xfffffffffffffffeULL > + > +#define MSR_HV_SIEFP 0x000A0012 > +#define MSR_HV_SIEFP_ENABLE 0x0001ULL > +#define MSR_HV_SIEFP_RSVD_MASK 0x0ffeULL > +#define MSR_HV_SIEFP_PGSHIFT 12 > + > +#define MSR_HV_SIMP 0x000A0013 > +#define MSR_HV_SIMP_ENABLE 0x0001ULL > +#define MSR_HV_SIMP_RSVD_MASK 0x0ffeULL > +#define MSR_HV_SIMP_PGSHIFT 12 > + > +#define MSR_HV_EOM 0x000A0014 > + > +#define MSR_HV_SINT0 0x000A0000 > +#define MSR_HV_SINT_VECTOR_MASK 0x00ffULL > +#define MSR_HV_SINT_RSVD1_MASK 0xff00ULL > +#define MSR_HV_SINT_MASKED 0x00010000ULL > +#define MSR_HV_SINT_AUTOEOI 0x00000000ULL > +#define MSR_HV_SINT_RSVD2_MASK 0xfffffffffffc0000ULL > +#define MSR_HV_SINT_RSVD_MASK (MSR_HV_SINT_RSVD1_MASK = | \ > + MSR_HV_SINT_RSVD2_MASK) > + > +#define MSR_HV_STIMER0_CONFIG 0x400000b0 > +#define MSR_HV_STIMER_CFG_ENABLE 0x0001ULL > +#define MSR_HV_STIMER_CFG_PERIODIC 0x0002ULL > +#define MSR_HV_STIMER_CFG_LAZY 0x0004ULL > +#define MSR_HV_STIMER_CFG_AUTOEN 0x0008ULL > +#define MSR_HV_STIMER_CFG_SINT_MASK 0x000f0000ULL > +#define MSR_HV_STIMER_CFG_SINT_SHIFT 16 > + > +#define MSR_HV_STIMER0_COUNT 0x400000b1 > + > +/* > + * CPUID leaves > + */ > + > +#define CPUID_LEAF_HV_MAXLEAF 0x40000000 > + > +#define CPUID_LEAF_HV_INTERFACE 0x40000001 > +#define CPUID_HV_IFACE_HYPERV 0x31237648 /* HV#1 = */ > + > +#define CPUID_LEAF_HV_IDENTITY 0x00000100 > + > +#define CPUID_LEAF_HV_FEATURES 0x00000200 > +/* EAX: features include/hyperv.h CPUID_HV_MSR */ > +/* ECX: power management features */ > +#define CPUPM_HV_CSTATE_MASK 0x000f /* deepest C-state */ > +#define CPUPM_HV_C3_HPET 0x0010 /* C3 requires HPET */ > +#define CPUPM_HV_CSTATE(f) ((f) & CPUPM_HV_CSTATE_MASK) > +/* EDX: features3 */ > +#define CPUID3_HV_MWAIT 0x0001 /* MWAIT */ > +#define CPUID3_HV_XMM_HYPERCALL 0x0010 /* Hypercall = input through > + * XMM regs */ > +#define CPUID3_HV_GUEST_IDLE 0x0020 /* guest idle */ > +#define CPUID3_HV_NUMA 0x0080 /* NUMA distance = query */ > +#define CPUID3_HV_TIME_FREQ 0x0100 /* timer frequency query > + * (TSC, LAPIC) */ > +#define CPUID3_HV_MSR_CRASH 0x0400 /* MSRs for guest crash = */ > + > +#define CPUID_LEAF_HV_RECOMMENDS 0x00000201 > +#define CPUID_LEAF_HV_LIMITS 0x40000005 > +#define CPUID_LEAF_HV_HWFEATURES 0x40000006 > + > +/* > + * Hyper-V Monitor Notification Facility > + */ > +struct hyperv_mon_param { > + uint32_t mp_connid; > + uint16_t mp_evtflag_ofs; > + uint16_t mp_rsvd; > +} __packed; > + > +/* > + * Hyper-V message types > + */ > +#define HYPERV_MSGTYPE_NONE 0 > +#define HYPERV_MSGTYPE_CHANNEL 1 > +#define HYPERV_MSGTYPE_TIMER_EXPIRED 0x80000010 > + > +/* > + * Hypercall status codes > + */ > +#define HYPERCALL_STATUS_SUCCESS 0x0000 > + > +/* > + * Hypercall input values > + */ > +#define HYPERCALL_POST_MESSAGE 0x005c > +#define HYPERCALL_SIGNAL_EVENT 0x005d > + > +/* > + * Hypercall input parameters > + */ > +#define HYPERCALL_PARAM_ALIGN 8 > +#if 0 > +/* > + * XXX > + * <> requires > + * input parameters size to be multiple of 8, however, many post > + * message input parameters do _not_ meet this requirement. > + */ > +#define HYPERCALL_PARAM_SIZE_ALIGN 8 > +#endif > + > +/* > + * HYPERCALL_POST_MESSAGE > + */ > +#define HYPERCALL_POSTMSGIN_DSIZE_MAX 240 > +#define HYPERCALL_POSTMSGIN_SIZE 256 > + > +struct hypercall_postmsg_in { > + uint32_t hc_connid; > + uint32_t hc_rsvd; > + uint32_t hc_msgtype; /* HYPERV_MSGTYPE_ */ > + uint32_t hc_dsize; > + uint8_t hc_data[HYPERCALL_POSTMSGIN_DSIZE_MAX]; > +} __packed; > +CTASSERT(sizeof(struct hypercall_postmsg_in) =3D=3D = HYPERCALL_POSTMSGIN_SIZE); > + > +/* > + * HYPERCALL_SIGNAL_EVENT > + * > + * struct hyperv_mon_param. > + */ > + > +#endif /* !_HYPERV_REG_H_ */ > diff --git a/sys/dev/hyperv/vmbus/aarch64/vmbus_aarch64.c = b/sys/dev/hyperv/vmbus/aarch64/vmbus_aarch64.c > new file mode 100644 > index 000000000000..ba4d8c9bc36e > --- /dev/null > +++ b/sys/dev/hyperv/vmbus/aarch64/vmbus_aarch64.c > @@ -0,0 +1,314 @@ > +/*- SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * Copyright (c) 2009-2012,2016-2017, 2022 Microsoft Corp. > + * Copyright (c) 2012 NetApp Inc. > + * Copyright (c) 2012 Citrix Inc. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice unmodified, this list of conditions, and the following > + * disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS = OR > + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED = WARRANTIES > + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE = DISCLAIMED. > + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, > + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES = (INCLUDING, BUT > + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS = OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON = ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR = TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE = USE OF > + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +/* > + * VM Bus Driver Implementation > + */ > + > +#include > +__FBSDID("$FreeBSD$"); > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include "acpi_if.h" > +#include "pcib_if.h" > +#include "vmbus_if.h" > + > +static int vmbus_handle_intr_new(void *); > + > +void vmbus_handle_timer_intr1(struct vmbus_message *msg_base, > + struct trapframe *frame); > +void vmbus_synic_setup1(void *xsc); > +void vmbus_synic_teardown1(void); > +int vmbus_setup_intr1(struct vmbus_softc *sc); > +void vmbus_intr_teardown1(struct vmbus_softc *sc); > + > +void > +vmbus_handle_timer_intr1(struct vmbus_message *msg_base, > + struct trapframe *frame) > +{ > + // do nothing for arm64, as we are using generic timer > + return; > +} > + > +static int > +vmbus_handle_intr_new(void *arg) > +{ > + vmbus_handle_intr(NULL); > + return (FILTER_HANDLED); > +} > + > +void > +vmbus_synic_setup1(void *xsc) > +{ > + return; > +} > + > +void > +vmbus_synic_teardown1(void) > +{ > + return; > +} > + > +int > +vmbus_setup_intr1(struct vmbus_softc *sc) > +{ > + int err; > + struct intr_map_data_acpi *irq_data; > + > + sc->ires =3D = bus_alloc_resource_any(device_get_parent(sc->vmbus_dev), > + SYS_RES_IRQ, &sc->vector, RF_ACTIVE | RF_SHAREABLE); > + if (sc->ires =3D=3D NULL) { > + device_printf(sc->vmbus_dev, "bus_alloc_resouce_any = failed\n"); > + return (ENXIO); > + } else { > + device_printf(sc->vmbus_dev, "irq 0x%lx, vector %d end = 0x%lx\n", > + (uint64_t)rman_get_start(sc->ires), sc->vector, > + (uint64_t)rman_get_end(sc->ires)); > + } > + err =3D bus_setup_intr(sc->vmbus_dev, sc->ires, INTR_TYPE_MISC, > + vmbus_handle_intr_new, NULL, sc, &sc->icookie); > + if (err) { > + device_printf(sc->vmbus_dev, "failed to setup IRQ %d\n", = err); > + return (err); > + } > + irq_data =3D (struct intr_map_data_acpi = *)rman_get_virtual(sc->ires); > + device_printf(sc->vmbus_dev, "the irq %u\n", irq_data->irq); > + sc->vmbus_idtvec =3D irq_data->irq; > + return 0; > +} > + > +void > +vmbus_intr_teardown1(struct vmbus_softc *sc) > +{ > + int cpu; > + > + sc->vmbus_idtvec =3D -1; > + bus_teardown_intr(sc->vmbus_dev, sc->ires, sc->icookie); > + > + CPU_FOREACH(cpu) { > + if (VMBUS_PCPU_GET(sc, event_tq, cpu) !=3D NULL) { > + taskqueue_free(VMBUS_PCPU_GET(sc, event_tq, = cpu)); > + VMBUS_PCPU_GET(sc, event_tq, cpu) =3D NULL; > + } > + if (VMBUS_PCPU_GET(sc, message_tq, cpu) !=3D NULL) { > + taskqueue_drain(VMBUS_PCPU_GET(sc, message_tq, = cpu), > + VMBUS_PCPU_PTR(sc, message_task, cpu)); > + taskqueue_free(VMBUS_PCPU_GET(sc, message_tq, = cpu)); > + VMBUS_PCPU_GET(sc, message_tq, cpu) =3D NULL; > + } > + } > +} > +/*- SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * Copyright (c) 2009-2012,2016-2017, 2022 Microsoft Corp. > + * Copyright (c) 2012 NetApp Inc. > + * Copyright (c) 2012 Citrix Inc. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice unmodified, this list of conditions, and the following > + * disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS = OR > + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED = WARRANTIES > + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE = DISCLAIMED. > + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, > + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES = (INCLUDING, BUT > + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS = OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON = ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR = TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE = USE OF > + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +/* > + * VM Bus Driver Implementation > + */ > + > *** 125 LINES SKIPPED *** >=20