From nobody Mon May 06 20:27:19 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VYCdG5mpzz5JGXx for ; Mon, 06 May 2024 20:27:30 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VYCdF6Z4Vz4F3B for ; Mon, 6 May 2024 20:27:29 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=freebsd.org (policy=none); spf=softfail (mx1.freebsd.org: 2001:638:a02:a001:20e:cff:fe4a:feaa is neither permitted nor denied by domain of tuexen@freebsd.org) smtp.mailfrom=tuexen@freebsd.org Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:3535:9fa8:93f9:e336]) (Authenticated sender: micmac) by drew.franken.de (Postfix) with ESMTPSA id 0483E721E280D; Mon, 6 May 2024 22:27:19 +0200 (CEST) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Re: Loading vmm From: tuexen@freebsd.org In-Reply-To: <987CD248-9FFD-46C2-A801-7C78A009D147@freebsd.org> Date: Mon, 6 May 2024 22:27:19 +0200 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: References: <760D76E5-F2D5-4D0D-9CDC-A42D3335C9DA@freebsd.org> <08796868-9E24-4BFE-AE96-F8B4A7FB6C0A@mit.edu> <987CD248-9FFD-46C2-A801-7C78A009D147@freebsd.org> To: John F Carr X-Mailer: Apple Mail (2.3774.500.171.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.36 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.962]; MV_CASE(0.50)[]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : No valid SPF, No valid DKIM,none]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; FROM_NO_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:680, ipnet:2001:638::/32, country:DE]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[tuexen]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; R_SPF_SOFTFAIL(0.00)[~all]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; TO_DN_ALL(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4VYCdF6Z4Vz4F3B > On 14. Apr 2024, at 17:05, tuexen@freebsd.org wrote: >=20 >> On 14. Apr 2024, at 14:17, John F Carr wrote: >>=20 >> See bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D277559. = It was suspected to be specific to the heterogenous Rockchip RK3399 but = perhaps it is more widespread. >>=20 >> When I first hit the bug there was a 30-50% chance that the first = kldload vmm would hang the system so hard I couldn't even invoke the = debugger with break on a serial console. Later, with no relevant = changes to source code, it would take about 2,000 load/unload pairs to = hang the system. If that 1-in-2000 chance is typical this bug could = affect all systems. Give it a try on your 128 core system: >>=20 >> # while kldload vmm ; do echo -n . ; kldunload vmm ; done > I tried 70000 times on the 128 core system: No problem. >=20 > On the 32 core system the problem occurs every time I call kldload = vmm. > Tried 5 times, happened 5 times. >=20 > =46rom my point of view, the behavior is deterministic on both = systems. I just tried it again with the sources of today. On the 128 core system I can kldload vmm and use virtual machines just fine. However, on the 32-bit ampere system the system becomes completely unresponsive everytime I do kldload vmm and requires a reboot. Does anyone else has the same problem on an Ampere eMAG system? Best regards Michael >=20 > Best regards > Michael >>=20 >> If anybody can tell me how to debug at the hardware level using JTAG, = I can give it a try on my RockPro64 and see where the kernel is stuck. = I have used hardware level debugging tools, but they were always black = boxes my employer bought with instructions included. >>=20 >>> On Apr 14, 2024, at 04:50, tuexen@freebsd.org wrote: >>>=20 >>> Dear all, >>>=20 >>> using the sources for kernel/world of yesterday running >>> kldload vmm >>> on a 128 core Ampere system runs just fine: >>>=20 >>> CPU 0: ARM Neoverse-N1 r3p1 affinity: 18 0 0 >>> Cache Type =3D <64 byte D-cacheline,64 byte = I-cacheline,PIPT ICache,64 byte ERG,64 byte CWG,IDC> >>> Instruction Set Attributes 0 =3D = >>> Instruction Set Attributes 1 =3D >>> Instruction Set Attributes 2 =3D <> >>> Processor Features 0 =3D = >>> Processor Features 1 =3D >>> Memory Model Features 0 =3D = >>> Memory Model Features 1 =3D >>> Memory Model Features 2 =3D >>> Debug Features 0 =3D >>> Debug Features 1 =3D <> >>> Auxiliary Features 0 =3D <> >>> Auxiliary Features 1 =3D <> >>> AArch32 Instruction Set Attributes 5 =3D = >>> AArch32 Media and VFP Features 0 =3D >>> AArch32 Media and VFP Features 1 =3D >>>=20 >>> However, doing the same on a 32 core Ampere system results in >>> the system becoming unresponsive. No reaction on the console, >>> no message there, no response over the network. >>>=20 >>> CPU 0: APM eMAG 8180 r3p2 affinity: 0 0 >>> Cache Type =3D <64 byte D-cacheline,64 byte = I-cacheline,PIPT ICache,64 byte ERG,64 byte CWG> >>> Instruction Set Attributes 0 =3D >>> Instruction Set Attributes 1 =3D <> >>> Instruction Set Attributes 2 =3D <> >>> Processor Features 0 =3D >>> Processor Features 1 =3D <> >>> Memory Model Features 0 =3D = >>> Memory Model Features 1 =3D <8bit VMID> >>> Trying to mount root from ufs:/dev/ada0p2 [rw]... >>> Memory Model Features 2 =3D <32bit CCIDX,48bit VA> >>> Debug Features 0 =3D >>> Debug Features 1 =3D <> >>> Auxiliary Features 0 =3D <> >>> Auxiliary Features 1 =3D <> >>> AArch32 Instruction Set Attributes 5 =3D = >>> AArch32 Media and VFP Features 0 =3D >>> AArch32 Media and VFP Features 1 =3D >>>=20 >>> Any idea, what is going wrong? >>>=20 >>> Best regards >>> Michael >>=20 >>=20 >=20 >=20