From nobody Sat Apr 30 08:46:07 2022 X-Original-To: arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8D2951AC6537 for ; Sat, 30 Apr 2022 08:46:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-54.consmr.mail.gq1.yahoo.com (sonic308-54.consmr.mail.gq1.yahoo.com [98.137.68.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Kr2yP2Bbyz4hQy for ; Sat, 30 Apr 2022 08:46:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1651308373; bh=CcZx+Zuui3vrgWy5W0hqnn2Dn6YHDV9EbtVr24PryjQ=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=gjvVNQN/NhKqiBH9438Q4iZF//a3xTx6u5IPubZbL7EQXg7vlOLNEPehTcTSqSytwO1hHquI9Sgz89lD5CkfDTjlaEcRJy4z6vhS/VXzIFyLgSa806jrurUMi8e1SHWlwNkkApAD8/W+L8XC5wD85XtDs7mHrvowYbwFJnt8obmLm1kR9vq5usqamMKAG/ZxbVBisFpPw+0kj8A7LXDcooiHasNTJaL1pwRFFQC5tUz6SX6Q6Wo0OwkpnnSqwcbitXRBJmkU2iV4m5u3E/eJKT1Luj5SNanWmBRN5HHi8zsZlJFQaUeqQYVKfqoTj2rFIArBPlr1di4N5Q1duJ0YdA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1651308373; bh=tkQ4d41FkHwdFTFnr9oUgaXyWM7s2WGI91t1An9s8SZ=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=AsisUmEO6zuCp9Z1MEVZzz2rAV98tcgSnhwqlewJqeY5K9s9pYByGDF9MILnn6qw8JyHn5WRjN803kaCDt+X4BtceqkC2maGr2Zpws9qL64sG//iaEW4FuymmeopvssRsOdij7qCwOxZOxo3cE4+2XjBC5VCzMlGjJFCIGiKs+vJwI0meXXGASPhNIxXzZBUCdhqmupL9lbWsKe7hV+ImVzgsYlo8ae4csueOMjYpqSlJ09rwpGnEBRiwTftcw5k+FQ2hdDtH17G9dTB4dn1O28oIO4ZDX6jhZkJ6vmzywXBKCK2N0jMwGTm0xvgPO8HAsZRtW9YOU80fGjVH/MpGw== X-YMail-OSG: wPryvp4VM1neXAeH5nATSYspDni25SXAC5CoBLNSmPq1Q7DjTJ6av1DlEzbWpx6 lrlzJ.tAqxqeLgWcvgsTKT_fTSOiYhCDcgw1hZC8iPgm3g8PqI5JL.JPQZwzaY3tVvjPc7iojvb7 hQKxuKSvQ8UeKg8OHtMjbG_jCD7vgRCYsWCYncWehyJNKEcejNqGnR5z8hNNXpjP7UPRkgS8cvPn t_uKz4r1IfWeXh1zBVlBNbkqEF__DZxzbAQV6gzrHphR0DadMdItgd9IirznALANpdCAJ_JUQQA7 z4SUKt6Ysl6S1Lc3bE69RZfCZJOeefnCpeIOxNDSFGm.mt9NQTZgtJ3tb4vozVbfvPDW.PCRX2Cs gtxFaiXKchoMojAop5iQrkabSgi30gCiqqXg8V4dvzke0L_Tkf6uzwBoIcbquwigkt29qzuuSmPy 2RlEoMMqGeupkunsRpU8Ms.CTAZdAWW.aZBA9k1cgpIZbK4.lWLvkNrgpf0.lNviJmxcfVoPB3cf Ed7G_hNCV4F3bAuVzl.kNe6geikkhodQq3qbkIMqqEmcktcyCeXOAx57kK4PdYHEUXMwgpi53yK7 ty88fiVjIOCNQRefgNRG4620BeEfaMZ4GLk070mG.rTK2rPlN6k6ZzHk8JUdaCRvMceqhwRD7Wjj RaOu4nEJyuKlUoJ94fKvKp98bCEwi8NV.iZlBr9VnzS33f4F0DoaAazqjLfIfY937hy6K_PMBeoL 2ODPfRkcEveMqSg6eGg.hgeXR1XiF1FqYO1uvcfQvfmSxvYA_xNUZpRze34JlCQ.IBj9tQ9eiiCL PqOWVSWp2de.6lT1WNOBG0HKdA7fmIXxXJ803hKrUPq6_rGpOevkZ1TYvl7Ch0.5fxa6QmNRvy17 Xo1yEBM9sKdxB.mkvOhg1TIFlN2VV06YextsJsgG1yXrIOpddzU9ko_IG_CuPpsF6.zjdNQKZs99 MmtqiNrvoqAPwdgbs.N.UHn1D4XH0ZVDFfI6kXgj6_Od7tX3hQYtritFk0ytrqv0jkyWxYTxYy.v j0PbNFZ5PW0hoJkDwumqVAJnNocu_3.quYugFquGxC8ouhAvLmf2Sm4BKBi4aJTR6q6mTsgXQ2zR DA3yF1HftEmOdeWaNPCRLKevEUG4kKdVNW2HaguLhlJZy_GGqm8zS6OdRBdisvO8rDvkKYYFddUS aJLpbHea7hpZ5bRK.suLttAIPQEfzi2NAXW.f.zIx4NO3LzPMC5tQNhEWXDe7Ne7nAOAr_9_w3VQ 8IT5ZBKDDI6a1qAgc3fJ6pmL9Xhds6VIBji_Qcwhv9_the0JXlTLkrWUjxPRavEG69aI6Vk7vKMO nxmFdEaFQY4WzJ6_hdNygbTaCgIAf1tkvwf4EgTXRWIq5DtP6CqRQJ2NecSe4_6XslJTQmbMmbnw igxiKnYvn7fXas1U5ClNayGsIGYI0Gi_dm7GBEOPQKRjBpEwG3mhH8CBkek4GJPxrrfy.8UdW1cn EKaXZREoY7zlQzOSvZfzRc21WDbJ44kQkU.xzD4N1gldHyF8zUtOMt2OV2FPjkS7Xf_e93xOXk4L wB_BFYNMVGXw4EX88tqr_J7UJJoikaYYRZ13yix7ReroFUT8CJxTjldoxFatW63EF0H_LULqK3Xu KfBV3ro2eVBTf_tewIXhdVJKFXeFkOyfmewvbt9IQyaESo3td.dEQPDYMbhP0FNqnv4QWkDRDEBS Gqe5ySM9o.WHcBDCYr7WB79biyMBsEr_RhvlyeEq1Sexw4L_A4naxSbpdQUnHbOyMvWxs58IRlV3 hTTLe8xY0rEvu4cfQusIxf1Vjee8dqVuQDbjdqDNCxH2JaV7wKAUI2a6wynj2eFyc53W3BZHrncE fcwqEmD17f1rTMux_ngsL3VxhG2f0eufRESzHKlsNSsnWoYdvulncWdnaXCQ1RGucYBmhHMCxZqY MCcksbgSM3ii_AGaXFmGSbCOOJK8Pi9Us0qzHHPbcS4u7JQ4zk4FosGKpM68ZJIQjs_MdCQlj.R3 VKXeGKzlzPM.V5CBlPnL.vX5kyrPWGSMPSXU0BO5_hjen8E2J8Ru5f3WwcR4R6VsQrizs90UUQDf z_VcJZZFSq7TXOul4hl5qaibgMEzjLbKRrv2xd4kzHgTfUnLF12LBk8fOVC6pZwco5.GzgcvXcjk x4VT86HSKZAS9I_.wGKaLpiuuPhPfF0neTHS1czxIFN8wJPOCYTa6Dtu5PX5yzRdokc0QYHHEfcS NqQ1mNREThAo.eCqhz_0AaCeh3hcTBc1S1vfVzyrw3dELtdQcMDMkBDl5nNuhNu_JxNJtCveCRa1 J.kNzNIfZ.A-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Sat, 30 Apr 2022 08:46:13 +0000 Received: by hermes--canary-production-gq1-647b99747d-xkjwp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 8cdc0e4fe5c0bd8f4a294e9d185fe0f8; Sat, 30 Apr 2022 08:46:08 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: FYI: RPi* firmware tagged 1.20210805 *is* the last to be bootable by FreeBSD via fdt use; sequence of 2 failure modes after that Date: Sat, 30 Apr 2022 01:46:07 -0700 References: <836019DE-531E-4B49-8A82-0D8F84885C21@yahoo.com> <8291972B-A953-4FD9-AA67-9F1F15AA3940@yahoo.com> To: "freebsd-arm@freebsd.org" In-Reply-To: Message-Id: <4231C088-0156-4BFF-8B7E-BEBE76CB15B5@yahoo.com> X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4Kr2yP2Bbyz4hQy X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="gjvVNQN/"; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.30 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-1.69 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-0.96)[-0.964]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.30:from]; NEURAL_SPAM_SHORT(0.78)[0.777]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MLMMJ_DEST(0.00)[arm]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-ThisMailContainsUnwantedMimeParts: N [1.20210805 works if I avoid deleting a file that I should not have deleted.] On 2022-Apr-27, at 23:47, Mark Millard wrote: > [Just an FYI: I got ahold of the RPi3B and discovered that > it was not bootable via RPi* firmware tagged 1.20210805 . Turns out I had deleted a required firmware file from 1.20210805 without noticing. I finally noticed that I'd done so after ending up with a 1.20210727 also did not work in the RPi3B --for the same stupid-operator-error reason. > . . . (junk text removed) . . . >=20 > [I've not added to the below and have removed the long text > block of RPi4B boot failure output.] >=20 > On 2022-Apr-24, at 05:36, Mark Millard wrote: >=20 >> [I may have also found what leads to the extra messages for >> the 2nd failure mode, an independent issue it turns out.] >>=20 >> On 2022-Apr-24, at 04:37, Mark Millard wrote: >>=20 >>> [I think I found the reason for the boot crash that is >>> a common failure to both failure modes. The 2nd mode >>> has other issues I've not analyzed.] >>>=20 >>> On 2022-Apr-23, at 23:45, Mark Millard wrote: >>>=20 >>>> The following is based on a microsd card with 13.1-RC4 on >>>> it were I'd previously substituted my U-Boot 2022.04 build >>>> and tested with the RPi* firmware that is in the 13.1-RC4 >>>> image. Here I've tried replacing the RPi* firmware and >>>> holding the rest constant. >>>>=20 >>>> The boot tests are on a 8 GiByte RPi4B Rev 1.14 with the >>>> B0T stepping. I've not been copying over the linux kernels, >>>> which they also bundle with the firmware. >>>>=20 >>>> [13.1-RC4 is just what I happened to use. I doubt anything >>>> here is special to 13.* or stable/13 or main [so: 14]. >>>> (I do not use 12.* or stable/12.)] >>>>=20 >>>> The observed status went like . . . >>>>=20 >>>>=20 >>>> firmware-1.20210805/boot/ >>>>=20 >>>> The RPi* release tagged 1.20210805 is the last version that >>>> FreeBSD booted with. (Other than booting, logging in, and >>>> shutting down, I've not been testing other aspects of >>>> operation.) >>>>=20 >>>> =46rom what I've read, firmware-1.20210805/boot/ should be >>>> recent enough to handle the Rev 1.15 related PMIC variation. >>>>=20 >>>> [I'll note that firmware build dates need not be the same day >>>> as the date encoded into the tag --in fact it is usually some >>>> earlier day. On rare occasion it can be a lot earlier, and >>>> there is an example of that below.] >>>>=20 >>>>=20 >>>> After firmware-1.20210805 there are 2 major failure modes. >>>> Both stop at the same sort of point in the messaging --but >>>> there is a huge difference in the count of earlier error >>>> messages. It looks to me like all the issues require >>>> FreeBSD changes if modern RPi* firmware/dtb's are to be >>>> usable via fdt. >>>=20 >>> I've noticed a difference between the working context and >>> the failing ones (both failure modes). >>>=20 >>> Failing: >>>=20 >>> spi0: mem 0x7e204000-0x7e2041ff irq 18 = on simplebus0 >>> spibus0: on spi0 >>> spibus0: at cs 0 mode 0 >>> spibus0: at cs 1 mode 0 >>> NOTE BELOW LINES MISSING HERE. >>> sdhci_bcm0: mem = 0x7e300000-0x7e3000ff irq 24 on simplebus0 >>>=20 >>> Working: >>>=20 >>> spi0: mem 0x7e204000-0x7e2041ff irq 18 = on simplebus0 >>> spibus0: on spi0 >>> spibus0: at cs 0 mode 0 >>> spibus0: at cs 1 mode 0 >>> START LINES MISSING ABOVE >>> iichb0: mem 0x7e804000-0x7e804fff irq = 26 on simplebus0 >>> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 30,31,32,33,34,35,36,37,38,39,40 on simplebus0 >>> bcmwd0: mem = 0x7e100000-0x7e100113,0x7e00a000-0x7e00a023,0x7ec11000-0x7ec1101f on = simplebus0 >>> bcmrng0: mem 0x7e104000-0x7e104027 on = simplebus0 >>> gpioc1: on gpio1 >>> END LINES MISSING ABOVE >>> sdhci_bcm0: mem = 0x7e300000-0x7e3000ff irq 73 on simplebus0 >>>=20 >>> In particular: >>>=20 >>> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 30,31,32,33,34,35,36,37,38,39,40 on simplebus0 >>>=20 >>> being missing means no bcm_dma_attach and that in turn means >>> that the static bcm_dma_sc =3D=3D NULL still. >>>=20 >>> The panic was: panic: vm_fault failed: ffff000000862134 >>>=20 >>> where: >>>=20 >>> ffff000000862134 ldaxr x1, [x9] >>>=20 >>> which is part of: >>>=20 >>> int >>> bcm_dma_allocate(int req_ch) >>> { >>> struct bcm_dma_softc *sc =3D bcm_dma_sc; >>> int ch =3D BCM_DMA_CH_INVALID; >>> int i; >>>=20 >>> if (req_ch >=3D BCM_DMA_CH_MAX) >>> return (BCM_DMA_CH_INVALID); >>>=20 >>> /* Auto(req_ch < 0) or CH specified */ >>> mtx_lock(&sc->sc_mtx); >>> . . . >>>=20 >>> So the likes of &sc->sc_mtx end up being a small offset >>> from address zero: >>>=20 >>> x9: 20 >>>=20 >>> Thus the panic. >>>=20 >>> As to how bcm_dma_allocate happened without bcm_dma_attach >>> happening first . . . >>>=20 >>> The working context's dtb has the ordering: >>> (I also show mmcnr@ and the brcm,bcm2711-dma >>> just for reference.) >>>=20 >>> dma@7e007000 { >>> compatible =3D "brcm,bcm2835-dma"; >>> . . . >>> mmc@7e300000 { >>> compatible =3D "brcm,bcm2835-mmc", = "brcm,bcm2835-sdhci"; >>> . . . >>> mmcnr@7e300000 { >>> compatible =3D "brcm,bcm2835-mmc", = "brcm,bcm2835-sdhci"; >>> . . . >>> dma@7e007b00 { >>> compatible =3D "brcm,bcm2711-dma"; >>>=20 >>> But the failing context's dtb has the ordering: >>> (I also show mmcnr@ and the brcm,bcm2711-dma >>> just for reference.) >>>=20 >>> mmc@7e300000 { >>> compatible =3D "brcm,bcm2835-mmc", = "brcm,bcm2835-sdhci"; >>> . . . >>> dma@7e007000 { >>> compatible =3D "brcm,bcm2835-dma"; >>> . . . >>> mmcnr@7e300000 { >>> compatible =3D "brcm,bcm2835-mmc", = "brcm,bcm2835-sdhci"; >>> . . . >>> dma@7e007b00 { >>> compatible =3D "brcm,bcm2711-dma"; >>>=20 >>> So, for sequential handling in the failing case, the dma@7e007000 >>> would use bcm_dma_allocate before the bcm_dma_probe/bcm_dma_attach >>> sequence had happened, leading to the crash. >>>=20 >>> Note: I used "fdt print /" from U-Boot to get the dtb and its >>> ordering. This was based on the address that the RPi* firmware >>> reports when debugging output is enabled (0x4000 here). >>>=20 >>>=20 >>>> The 1st mode happens for (I've added the -fails notation): >>>>=20 >>>> firmware-1.20210831-fails/boot/ >>>> firmware-1.20210928-fails/boot/ >>>> firmware-1.20211007-fails/boot/ >>>> firmware-1.20211029-fails/boot/ >>>> firmware-1.20211118-fails/boot/ >>>> firmware-1.20220308_buster-fails/boot/ >>>> (The _buster one has firmware from 2021-Dec-01, which >>>> is before all the tagged releases listed below. >>>> It looks like the switch to the new major kernel >>>> version after buster came with other changes that >>>> FreeBSD has not tracked.) >>>>=20 >>>>=20 >>>> The 2nd mode happens for the following. (Again with extra >>>> notation.) There are a lot more error messages before the >>>> panic happens for these. The firmware builds for these >>>> are more recent than for the above list. >>>>=20 >>>>=20 >>>> firmware-1.20220118-fails/boot/ >>>>=20 >>>> firmware-1.20220120-fails/boot/ >>>> firmware-1.20220308-fails-non-kernels-same-as-1.20220120/boot/ >>>> (I did not repeat the testing of the unchanged firmware. >>>> I just did the "diff -r" to discover the lack of change.) >>>>=20 >>>> firmware-1.20220328-fails/boot/ >>>> = firmware-1.20220331-fails-non-kernels-same-as-firmware-1.20220328-but-for-= bcm2711-dtb-files/boot/ >>>> (Since the .dtb for the RPi4B was different, I did test this.) >>=20 >> It looks like the extra messages, blocks of: >>=20 >> clk_fixed4: disabled on ofwbus0 >> clk_fixed4: Cannot FDT parameters. >> device_attach: clk_fixed4 attach returned 6 >>=20 >> Are tied to new dtb content in 2022's dtb updates: >>=20 >> cam1_clk { >> compatible =3D "fixed-clock"; >> #clock-cells =3D <0x00000000>; >> status =3D "disabled"; >> phandle =3D <0x000000e2>; >> }; >> . . . >> cam0_clk { >> compatible =3D "fixed-clock"; >> #clock-cells =3D <0x00000000>; >> status =3D "disabled"; >> phandle =3D <0x000000e4>; >> }; >>=20 >> These 2 did not exist back when the 1st failure mode >> started. They appear to be repeatedly processed from >> not really being handled --leading to lots of >> messages. >>=20 >> The messages may just be noise for activity that is >> not contributing to boot failures at all. So fixing >> what I called the 1st failure mode might actually fix >> booting for all the firmware versions after the >> version tagged 1.20210805 . >>=20 >>>> The failures look like (each test shown) . . . >>>>=20 >>>>=20 >>>> . . . >>>=20 >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com