From nobody Sun Apr 14 12:17:22 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VHTnx0Nvxz5GSvK for ; Sun, 14 Apr 2024 12:17:25 +0000 (UTC) (envelope-from jfc@mit.edu) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2139.outbound.protection.outlook.com [40.107.236.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VHTnw4v41z434G; Sun, 14 Apr 2024 12:17:24 +0000 (UTC) (envelope-from jfc@mit.edu) Authentication-Results: mx1.freebsd.org; none ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ncMLvw0ptT/iZ+VR+7OxdQY5UZxEiG2qrFFq4u52AKmQki9aIW9tIC9GSEmGJspRyHip4InjsU50RFrNOMEXCApmiNAM8wd0mkh2S2CEd3PBeHwEBnmxY3oIO94txEOP7xC/EHJYqhQrkjQ5JOqiXW/zHjeI9AqNN5kzihb578ncNEXhupj4BIXFoq+GqN1cXPxgAMuNG/IIALI7Zk1DUUEielxrKV/yfh6X1c+xlBNn5gF9VU2RShlRJYkzf2NWV6EpjuBjURt5UM56QZjGWMr7u5sk1QV8yvliOchJMrkBLWNylJiJJOWsqvpGNgnBFJOeMUZQuM1512dv+/WCyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nPNlH3UQjaQNpg4Yr2DPkawzH6Cy6MpfJAcwHvfprg4=; b=iJ+xjJ944/DAXXsoLF5DC9LSwd9FdQCurFCi2fG+guk6RbJoqhGFB+ZKOIZMGo70TvihP6AgTskZzjfgzm3tkxCB8TMG+uZmY1s+JyI4GB7S2JRdqlOE6ddTuRBRXqgNoExvw7T4sPVUKVnA1HhqCm7vSa48L4HkNi5KMsc0W723RFBI86sKuoNTSq9zGoAEFJL4Nlqht8lY25ealVuNSHK4zjl4+JnEm4vhnyJw2OOVAeaulA8e5JA8DXCJVhg3KnvQtftqoXrTjRSm3/u7JqaR2GRuH0a6gFZC+Js9HKukv4boCJKSFaUnOie5jRQLaTdcTbmqyJUzynS0hzgUDw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mit.edu; dmarc=pass action=none header.from=mit.edu; dkim=pass header.d=mit.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nPNlH3UQjaQNpg4Yr2DPkawzH6Cy6MpfJAcwHvfprg4=; b=tGVLDKGQjbzzCWaQdfAohYCRqHP7CBMweAsHwZUFCc8CHAkQVGsYgJl6WUPEtZmWCEzJSnTvd1tnKalZYNR+58zwW/y/iSw2wbBGg/cBPZtGCtErL+SsZWF5SfobRA3o/VNCLl7aCW3g6ulQ7HQldILkG9cW0gMO8wASe82bjs8= Received: from SA3PR01MB8450.prod.exchangelabs.com (2603:10b6:806:382::17) by SN7PR01MB7971.prod.exchangelabs.com (2603:10b6:806:32b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7452.49; Sun, 14 Apr 2024 12:17:22 +0000 Received: from SA3PR01MB8450.prod.exchangelabs.com ([fe80::bb39:d8c:f575:6b9d]) by SA3PR01MB8450.prod.exchangelabs.com ([fe80::bb39:d8c:f575:6b9d%4]) with mapi id 15.20.7452.046; Sun, 14 Apr 2024 12:17:22 +0000 From: John F Carr To: "tuexen@freebsd.org" CC: freebsd-arm Subject: Re: Loading vmm Thread-Topic: Loading vmm Thread-Index: AQHajkjw/VnJDnh8FE2dbwilN/qqaLFnrsiA Date: Sun, 14 Apr 2024 12:17:22 +0000 Message-ID: <08796868-9E24-4BFE-AE96-F8B4A7FB6C0A@mit.edu> References: <760D76E5-F2D5-4D0D-9CDC-A42D3335C9DA@freebsd.org> In-Reply-To: <760D76E5-F2D5-4D0D-9CDC-A42D3335C9DA@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SA3PR01MB8450:EE_|SN7PR01MB7971:EE_ x-ms-office365-filtering-correlation-id: 96932ebd-3f70-4906-f9dd-08dc5c7cd647 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: VMiHFykvQGm49YcMFabpNlBejsfosKEn3SwFP0cJrxs16pBi33/PdlmSHze3E7kT1GvY+rY7ZLDHnFoA+TGNaC0rh0E/bkKGoh3wAnYN3ECDps/Za+aX2DXSXT6LRuY6wZNZEg4eFtEO9MBBY21+j9W96XMsXWmm1rf3l0jN57u+psPyAe5Lb4nxoV5gLcskcsY1kpIp+cDK0mUUr9hsUeCYYs1VVdNzEgk317SsGrOigU2kvDznTL/092BYXnif5lMnpIAIg9/nEG2iuCylVH/46HmuW8i/BDDzBe+P093u/DQZOyt60/BLmOyWJWJ0DtVxjj/U+ao3Baldxghlz4PGAtGCpHAIij8THL2dBdAsjDW1Gmc1u9yklfyvDyivipzVLvwrqZ2xLxFTcoSQJcz2hYwOrqbYvIFCwicmkHRHh8mXrVxwqgBPVKlqjmpjNk3Swsn13vs4D527nNiLw0jXzQKMDDqps3doCjIcF7whgGlFbAs/nQanFAXQpjQoRWjBnPA/KVExbJMSkuU5ro3fmo0jR8hoFLZnSa6BfkoqiFLJUv5GiEczA0PS1B8DajcbgXWMNIXAx9uqhM0MhW0lIDgmgOcef1F2apL+PZkWwYftRH9I/NnVi1gC0rBEewmeQVGgW3sXhPwFN3A05a2CgOW7Hdc6Zmy2jnJwBu37JnqBjfTrhah4OlWKuteXRbHvYvxVaY5rL2nlGEj85tH1cmITHYzGz3t6AKKDDQ4= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA3PR01MB8450.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(376005)(38070700009);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?oNQy4vdXzkRJ5RGxi1rIT9eorvIm9gY4XvaO3zd8WcfAvpiapiiA42Zy+Tba?= =?us-ascii?Q?MhG/RHHCllJgT+Da0LqB+pdBVE5hwElbfVngCR/D8f9H6IRHaiTO2g2OooJ8?= =?us-ascii?Q?npKRiYW+4pRLzb57V553nSvkbiPEj/RgI3twLnEXYGnaQOXLQ3ndY3M9+VCb?= =?us-ascii?Q?PgbGLOtx+gGrIpcHD0ptD8ySY6Iln2V5cVgKYlnN8PZuUs1gjXBIJQ+xOdo5?= =?us-ascii?Q?tXwbTOfP2C75i7Q1vdJqY/hCfo4FLevLJ21S32C+ACDXWhCA3qgKwPe9Npwh?= =?us-ascii?Q?cIZSdHmgrosojkyi+nRo32xKOMSVDEeNy8Rx5z7206V438TAnTJ2UY1P7M3S?= =?us-ascii?Q?TLSZiraO2eSZQzM6RguI+MiaRul+nO/7Z4CrVP2/ikGevtdzA+dWaWLp9v0x?= =?us-ascii?Q?qCNerBM9irrEZoyJlo0bIzA096/MsWb1bL9DAjGPh5wH7kgCfpt9GAlb+dZ0?= =?us-ascii?Q?vAMCuDvVamsTB9sGTcz4Fx9h/huGKUy1I0geDKyXcAemx1FBTctpUvcvruT1?= =?us-ascii?Q?CENWg/0yua49/WxRD/tAGFXi/IEoNo40LU/gOhRGjvsmfhN4RmNM8qLD3kPq?= =?us-ascii?Q?LXAvbTQGEW2QAlRrjKeUNl5RFJ2t+nkNo1oCzDAFQfTI5MdOnmzzeo2xnvLg?= =?us-ascii?Q?8H3LUOi6fSPbJIAt9teHjeaG3ydr5VvUTEvxTX9ZpAu/L6kenujXg626AbCd?= =?us-ascii?Q?Un5y+5GNrslfqgCUYsYonRacwxPdcWZxqxGfZ656qZplkKT0X3T66ACnhLdw?= =?us-ascii?Q?km2DrYOzBcOrnOV51RZixSeFDC7aOnawOvEz9/BnGgN2iwdcOq3jZag/+5DT?= =?us-ascii?Q?E3xF5oigAyCHWo94K86wOad7XBz8sdrwCbioR5ran2+QGr4E7qqbLFkIkAD4?= =?us-ascii?Q?1Tt+g3ri707Gi5tO+mwJJDBU/1DyXeoezuVUwjC0V0zURMZVjGus2syXKM6M?= =?us-ascii?Q?icfy9ECWxmYmYt+P57dpzLOiQmHw1hQVlc8c333gd2KlGl81++6h/J49ua6x?= =?us-ascii?Q?/XWIr2Lp9pkCpj1FgkzJRDtqk3t2oqNUGnLdWj9ims37df9+ly9dtgDgijQ9?= =?us-ascii?Q?owD4BeoMqHSMa4JCVmkaWXZcefGNAl9KkdSf1vN9GV73gLYkWxwxXiz9dEAe?= =?us-ascii?Q?kgPsbITGi5Xf6HrLg7vBpkGymsDQe8F8Vj8viu2R/bzT4SbdsCwb98smfF3S?= =?us-ascii?Q?wt5kTlxPEW1Nftm2CyXCpaJoHRqjfdhYcTjeMMRdX0X/nI9crAcWjy2s2c/k?= =?us-ascii?Q?oZ2uCHgFlIs7b+q0XGvqt29OyKm0p84LIq8nWShU/Fs4FXhqeBntXANly6Tu?= =?us-ascii?Q?MgCnCu4UnWfE7F/NzXQ95nYJj34DC5XDvbZ3U9Y2gUbVSCrLb07L1iVJETUk?= =?us-ascii?Q?mAk2ivW7M/5DCr/QXDgvZT5r+KTgrYZG+kGp9f1tPmr3oICj4gRpAIcPmGF8?= =?us-ascii?Q?mjMre+yXJ8ofQz0MHib2PM+dZ1LrfLYG0WojsF/eeAZVQ4w7o/QpgV8w4k3K?= =?us-ascii?Q?svWGrmYLPebFTrmk9xNFFpDP0RR7ZErEEarfoNt5X1XcLqjKfe8EK/HLdukb?= =?us-ascii?Q?QL425EigHYVxiBH9zepWxlkus30BpxfuVAFvZZHTGbrX4BvAuoRZ/8tJn6go?= =?us-ascii?Q?D/YMRWmWFI9muQKYvUblHRUZcJMoDkD2NBQrzuJiH/dI?= Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-OriginatorOrg: mit.edu X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SA3PR01MB8450.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: 96932ebd-3f70-4906-f9dd-08dc5c7cd647 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Apr 2024 12:17:22.0276 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 64afd9ba-0ecf-4acf-bc36-935f6235ba8b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: t8tzij0s0VuOSy1ckMyWBfSGXtS7nmzlDIQBgvdGiIT8TxUtYQFTR2Xp5hjbgIV7 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR01MB7971 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US] X-Rspamd-Queue-Id: 4VHTnw4v41z434G See bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D277559. It was= suspected to be specific to the heterogenous Rockchip RK3399 but perhaps i= t is more widespread. When I first hit the bug there was a 30-50% chance that the first kldload v= mm would hang the system so hard I couldn't even invoke the debugger with b= reak on a serial console. Later, with no relevant changes to source code, = it would take about 2,000 load/unload pairs to hang the system. If that 1-= in-2000 chance is typical this bug could affect all systems. Give it a try= on your 128 core system: # while kldload vmm ; do echo -n . ; kldunload vmm ; done If anybody can tell me how to debug at the hardware level using JTAG, I can= give it a try on my RockPro64 and see where the kernel is stuck. I have u= sed hardware level debugging tools, but they were always black boxes my emp= loyer bought with instructions included. > On Apr 14, 2024, at 04:50, tuexen@freebsd.org wrote: >=20 > Dear all, >=20 > using the sources for kernel/world of yesterday running > kldload vmm > on a 128 core Ampere system runs just fine: >=20 > CPU 0: ARM Neoverse-N1 r3p1 affinity: 18 0 0 > Cache Type =3D <64 byte D-cacheline,64 byte I-cacheline,= PIPT ICache,64 byte ERG,64 byte CWG,IDC> > Instruction Set Attributes 0 =3D > Instruction Set Attributes 1 =3D > Instruction Set Attributes 2 =3D <> > Processor Features 0 =3D > Processor Features 1 =3D > Memory Model Features 0 =3D > Memory Model Features 1 =3D > Memory Model Features 2 =3D > Debug Features 0 =3D > Debug Features 1 =3D <> > Auxiliary Features 0 =3D <> > Auxiliary Features 1 =3D <> > AArch32 Instruction Set Attributes 5 =3D > AArch32 Media and VFP Features 0 =3D > AArch32 Media and VFP Features 1 =3D >=20 > However, doing the same on a 32 core Ampere system results in > the system becoming unresponsive. No reaction on the console, > no message there, no response over the network. >=20 > CPU 0: APM eMAG 8180 r3p2 affinity: 0 0 > Cache Type =3D <64 byte D-cacheline,64 byte I-cacheline,PI= PT ICache,64 byte ERG,64 byte CWG> > Instruction Set Attributes 0 =3D > Instruction Set Attributes 1 =3D <> > Instruction Set Attributes 2 =3D <> > Processor Features 0 =3D > Processor Features 1 =3D <> > Memory Model Features 0 =3D > Memory Model Features 1 =3D <8bit VMID> > Trying to mount root from ufs:/dev/ada0p2 [rw]... > Memory Model Features 2 =3D <32bit CCIDX,48bit VA> > Debug Features 0 =3D > Debug Features 1 =3D <> > Auxiliary Features 0 =3D <> > Auxiliary Features 1 =3D <> > AArch32 Instruction Set Attributes 5 =3D > AArch32 Media and VFP Features 0 =3D > AArch32 Media and VFP Features 1 =3D >=20 > Any idea, what is going wrong? >=20 > Best regards > Michael