From nobody Mon Jul 24 13:12:59 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R8gZl368jz4p6nN for ; Mon, 24 Jul 2023 13:13:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-20.consmr.mail.gq1.yahoo.com (sonic305-20.consmr.mail.gq1.yahoo.com [98.137.64.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4R8gZk06Hsz3x0g for ; Mon, 24 Jul 2023 13:13:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=AJQTFAQz; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1690204395; bh=AZP8jbWMXuvo3lUQbnYUuWG+dB553x99vr943NCvcAY=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=AJQTFAQzeFYgrPDlNWzTYmVq0hX6XHl8E8YjgSg81ZK1Qkj70CS55GhScfATmnYuLxkPvruOv//dK15jbryGYOhHk8NCSJUYLp5J2MvcLBpDalx8J/0xALjt9mUbD115nmHs2mKc+1HG/AgaAFm10RlN9t8DgCuH71YRRCCguVT+yhdbm3LF2lUnUULJHeWXbf8brSxeR2eo1oBhe2rZTyLTR/SiVPqVPPXyCQTyi4o8QpjJt3T2jDstVa1dH8ZOsuvo385UKukkH3VfK7aB2RKRr4PvJdKbVyc6xCLhqps4mt2btiJFpKOqEgQDjp2Cq9wPAuQBPLN8wWEiajid0w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1690204395; bh=9X691PDi5lWZcoQ0CtZcPT/HCJq81N+BKC1fTJUN5QF=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=pShkUvChDN7AGXPVKh7CEpJE1jWKgpM3xYHeYXfzUKC70HwyuL/veKpwrCAQTWYkiaspJlpwyMoCmWBYc6MGsU6XCLnLZmZGptrDE9TaOc4eHN206v0jaf/5RJbYLR+j8zDzoLZ2mCUC2VY0q172AbXJQ4rH8nqQ2IizTEF48FYIv+aeChO6qQhUdXfxD4r+HpOS0/unObuukpUrSUoju4aCM3sJSeJ026Wc8542r6T41T6kIv2+2XZiL2uw+OAamfOk04OiqY+N8THctww4/Sngv/aN8expAP90nTjzHAdumRyLfW/AMDRTeXOMUALrClBHUrlLaM/aczucAGzKHw== X-YMail-OSG: 9_4KDIIVM1n1lB2GbCBI9Q_E2U00elru1cV5utQIDavo1trk.17WFC1RcPICq4L ZaOZWCA6.GKuW3CDbgnL3LXxNbjpcYARbn9qQDfvBZa57IzuS1oQKg0uU9AvTvGYL7BnAHEMAZS6 lqBlN99P8GiW370DHlp6eHPKNLV2NIMxJBslcUETIe6r7Avmfv11PssxZQFfpCAZLgeWxBeAdkPC 4MKI7R14guE3nkBFkrrYtz62izd4tSibhtXqVMNDgAZ4R40p8PCvElWd2eXlA2S3t4jwvASkKcR2 lJqQfjKLr7iHH_M5ne8FsQspPIxg1EjFjURavJ3svumCY1WrdyZBIPyFT4thksN_6cezxAEaXnQG UZt.k0PmWu6Ly.L0YJGm9BRwOsEYwdagplEyS.SSgvdLiTerw.dqpAoCWNSbrXiPnurwV5YM8YcP 7nTlY_KUI9JfYpHTLUHlOtDsyXRyeeeDBquFFB9HEHFwp9yfI6zdlNv8c3Gm4Zcy0r.6Te4y4XKo ujLVXEljVmEYqnRit6DiVmiNvK0xiecmVeLMrNJPX9CU4xy1wNTp2PUlaJWo6tbi9yTRaa_DmCVL yD5XzfIkBcG_VFjagIFlUAH9oDxZW2Q0nkN7WtJgmnDNh_gSqJTBldtMSvf9HN.eeG29Y.iwWm8C iM4GdHBVcTiHFg2vdLK6h2TsfAYm.M0OJurTQZOAKHfIrey56BpvAlKiAUa3NpSYWdJx6D6ggGJB NCzSzc5UhlVAkIjMgBlFWt0fAy3vDUhdIb5l.chuk_TulbIrwIMa1EMAzmp_OTFxk8Vpp11GEFee yEVIf4lF7lBQ9KFfiyse3VrZOpA3EbO5m0vXkLrbbF45g_D8dT8pAlAwUUChocJCi1V3jHXSl0.f 5mfxyTR9GrYaruo2pEgAO.A1HLI3LLC5bkaw01WUq9ce0l0iPrl13fykIjDYQTzXmt7imprYdimD d4PE0531nBnpnlDkU5T.QvvN96L2jQ05g0eAWreC4UxPUCng_QuM7ioe0h0sSb5ynpbNa4UTH8cI VeQy3wWE9qhcNHKzBf7krwAU7rGmW2.e6qj6hvXqesIkvKTxEwKPir.ki9va_ybQKbD2yIiw2FWx 43ciSft31nAPxmFTrqPQupt4n6xhX0dcI5_InWdkiGS8WSMKcDP0kg4CzoOBIDL2ZfTcuFzb5gA0 kUZI.06nkhULUbOxvXmTDYanqzXfaJ7PhYEVicVm4Qj0bHhvdqQaGDOQbKnl6ntXXZE.g1p6oPRS LQfl5ynNic4RZ8JZ.iSJ1X1Q6OKwwpmBXhVvqYmYRhYUNKll1pouighABgh2nM71QMlWGlmtVyUJ G9bK960_gyAmyCBmAtsSiMgDLtglcMml09V_srcyXbrqqxXq8p6NReRA0_9Ct7aPm5gmwwrBeg_O fI6oo5D139DNdoYm2dElSSwdr5o37pehBREgbyZxkXe.hHpRf3ylxxRcp0zhRrmpfL48bdhhQlWb WAR_nHSa1u6uICtDqXP9R11_yNuTak4FG8hPGOw0gNLoLHddq9fR0r7TI4DjAYfJ9MpMf7WIr8fn GZOfy4JNT116CexyYor1fXSbogYVJy0yiWwlLbxcTY_GOHuGYpOWd8dncb1XA03VvXpmxhm8a_GH Qa5ci7_rq87LamsTJ_CP_H9T.VHRj_1enJ4_mvNIfrWodlnRKBFldLo5bDVFlb9HK6GQJ0EOx3yo W9OzxdM5BgGyDAmWjniSON9rI1Xvgq_I6V59rIBsm_hpC4NQ4PLMCLWY2z.lJE.hUTVe875eFdTi Gkna4r51BnpWPIZMlGsOQaenzcMMdJYbba45H_8y0sZvE2FK4X0Pnq56ge5qmI3mNEVsEFBh77oS uusX58fGsUTbpmF7JZsVBjnHEmYl2yln1U4zZEaZ6HY0oy3ElAC.392Cc1FDuMIT2n8oQcgBT9Y7 wt3Fq2jEbUKO2HLmRuloqeG9wi9BqfOIPuJT6twvxEqc2apN6kODWLJKYvjDRI4n6yODcV0.lqkc pcntrL9UnkXhvoDgwL1_YJSWiE86tEL7HwFo00itBb48hNtwSqTdlmlN3iluLL79hViK2Mq3P2Ap vG3ykHMa4aoUiiAtEShcJE_qAmLUFumVWWnUYNRfpzb3lhghEXkvFoq.uUYD_XzxzqXemn1zGTfg diGGnKfpHR78_c0RtZ.knNVjjflpwp7r2agivSjBEmf4GHelNYHv8w3gr1h8FWzhVAHvgB1t3yCy enrNS3f0K4ymla6KSlmaIVJh2LTVnno57jdqzpRKQLqVXVcuuoLalSF0u94AQY.eJ2x7wEOf6Ew- - X-Sonic-MF: X-Sonic-ID: 1db1630f-2fcd-4d6a-82ea-e11d6f91bddb Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Mon, 24 Jul 2023 13:13:15 +0000 Received: by hermes--production-gq1-767c8cd9fb-w5bhj (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 7a8eaffbec92b21ff9054d8e71bc6f88; Mon, 24 Jul 2023 13:13:10 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: git: 80e4ac2964a1 - main - Work around VNET and DPCPU related panics on aarch64 Message-Id: Date: Mon, 24 Jul 2023 06:12:59 -0700 To: Dimitry Andric , dev-commits-src-main@freebsd.org X-Mailer: Apple Mail (2.3731.600.7) References: X-Spamd-Result: default: False [-3.47 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-0.97)[-0.967]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.83:from]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[dev-commits-src-main@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.83:from] X-Rspamd-Queue-Id: 4R8gZk06Hsz3x0g X-Spamd-Bar: --- Dimitry Andric wrote on Date: Mon, 24 Jul 2023 11:06:54 UTC : > On 24 Jul 2023, at 00:48, Jessica Clarke wrote: > >=20 > > On 23 Jul 2023, at 23:38, Dimitry Andric wrote: > >>=20 > >> The branch main has been updated by dim: > >>=20 > >> URL: = https://cgit.FreeBSD.org/src/commit/?id=3D80e4ac2964a11edef456a15b77e43aad= eaf273a2 > >>=20 > >> commit 80e4ac2964a11edef456a15b77e43aadeaf273a2 > >> Author: Dimitry Andric > >> AuthorDate: 2023-07-23 13:48:36 +0000 > >> Commit: Dimitry Andric > >> CommitDate: 2023-07-23 22:35:04 +0000 > >>=20 > >> Work around VNET and DPCPU related panics on aarch64 > >>=20 > >> lld >=3D 14 and recent GNU ld can relax adrp+add and adrp+ldr > >> instructions, which breaks VNET and DPCPU when used in modules. > >=20 > > Thanks for committing the workaround. > >=20 > > This will need some kind of EN for 13.2 given LLVM 14 was merged in > > time for that and arm64 is a Tier 1 architecture in 13. > >=20 > > There perhaps also needs to be some serious thought into our testing > > and release procedures given we allowed a Tier 1 architecture to = have > > VNET and DPCU be totally broken in a point release for any kernel > > module. Especially when the bug was known, open against -CURRENT and > > triaged all before the MFC to stable/13; there needs to be better > > tracking of toolchain release blockers. >=20 > I agree, but how many users does this affect? I am only a sporadic = user > of aarch64 builds, but I never saw any panics, and apparently our CI > builders also do not see these. >=20 > So how often do these panics actually occur? Must you specifically use = a > VNET feature to encounter them? Prior to this change, if I build based on -mcpu=3Dcortex-a72 a linker = relax happens such that: # kldload if_epair.ko panics. At the stage of .o generation, the distinction is a minor variation in instruction ordering, where -mcpu=3Dcortex-a72 puts 2 relevant instructions next to each other that it otherwise had some of the other code between. The back-to-back pair ends up being relaxed by the linker. Use of -mcpu=3Dcortex-a53 does not put that instruction pair together: so no panic. Note that both builds are compatible with both processor types here. The -mcpu=3Dcortex-a72 code panics on both but the -mcpu=3Dcortex-a53 does not on both. -mcpu=3Dcortex-x1c allows newer instruction set additions. But its code generation did not put together the 2 instructions: no panics. But the Windows Dev Kit system that has the A78C/X1C cores does panic for the -mcpu=3Dcortex-a72 code. So: Minor code generation variations lead to the panics. I'll note that I discovered the panics via kyua runs with -mcpu=3Dcortex-a72 based builds on the various systems. I'd expect that the kyua runs should not lead to panics. I do not see "how many users does this affect?" as the major point of the reliability problem here. Future code generation changes for the same options are subject to having the problem. Future changes in compiler option use could have the same sort of result. Known VNET or DPCPU usage contexts are sufficient to have the problem. If there is a 3rd usage context as well, the VNET/DPCPU use would still be problems. In my view, having such a dependency on accidentally avoiding some minor variations in the code generation is not appropriate. Use of --no-relax (for now) at least appears to systematically avoid the dependency on the minor variations that happen to matter here, whatever its other consequences. (More linker activity than necessary may well be disabled.) I have re-enabled my -mcpu=3Dcortex-a72 use and tested the result: no panics. =3D=3D=3D Mark Millard marklmi at yahoo.com