From nobody Thu Apr 28 06:47:37 2022 X-Original-To: arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9E8401AB755B for ; Thu, 28 Apr 2022 06:47:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-25.consmr.mail.gq1.yahoo.com (sonic312-25.consmr.mail.gq1.yahoo.com [98.137.69.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4KpmQR3kNfz4YB3 for ; Thu, 28 Apr 2022 06:47:43 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1651128462; bh=AKsycAhDVzxeMZJDQsWbWVgOU91KR1LZH3RAwjah6xo=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=KSAXaDxiZo4I54XwsPtn1UWKr988G6A/vYLhoOxmt4B1lZ0NPMUAge/jvY1tD9lCpeFQh1Y51K/Z7FJ21gODecj8Xa4zjRL8IYohUfW5Q/pNQYQlVHntK1GZtW8GPZsAGFrG6oxglqHTPhFQAO3EH0QtSiwNEJqoP7frSyY8SvnmZ3kF3eSuPmv7vhUGOIyFq75vFCvCWNKHPS9PbUomAXmrKWTm0JtLSBeUjfQ006bdqw6MHLDNrQV8+uDwYha7NBNIiuIM+F39KgvsO7rL99CVknZuFHWNLbbQCPNFhVpSiFVV8c7U4DxmhxB3Fzy2gJhsAboZqPB2ZN/E8kYPAQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1651128462; bh=k3Ksl5vlZzm17zGylwg+7EVTNVImrn8r74lKv9kfPph=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=WoWwn/W3EsVEZJ2pR9eQJWCkmf01QNO2dxNB+5yJFAOzyJ/8FFnyvgb9s7JX01MwOtmS2sF9shkvLnECsvAmM8qCLwqojDO5lyz7/VCHOeeKMGN9jIhwRsht7w/KvCNxwpCgRw2FfqTi/9vnDwILI5uGk2RYzy4vSFifPJUxqL+RtXz2Unto+wqR4PAefRuhJx58MylcnzWbwdhXmDoLuk9n7Q80ZtFwHBY+j083dK+ba6E0BhXjUauURLficU85RdP80lY8eiYo/C41QlILySXw11zrOIQCJym7viZXVBGr9oxaPNv09DKeGwOOjQXnMhjxYM96wyNHbT+ozTOUhQ== X-YMail-OSG: IGd3hw8VM1nSAeZYqTbPth2GJNmS7ixYPANPsq5NWZwKcaGmhYOg_3F7vjxjO_e Yfru39z_qeHw9xdGI8EKnHAfDTqV5vUz3simgYRVi8Vyn1Iyo2oVy6TJAjlWPkU3et4wHOblwJF1 3vGk_sf9C08F9iJIq0z0oMIA4UmQjxstel3RlL5aFt6eVfZzDNSdQVmRenmmQr6YRoZ9SWzXWdBy 2SzDZMR48aoMUs1FcXBaKM0Yf6ZaZHlaOcbaqVoGqcefdp.SkyT2UFcR27iWNRbpKY8HtAcan7GC fIEFU1JMWM_h6XHKxsXJat_VZTp.jv4jjNZe3dMMXqSVFQKnePXCP94I6J8GBe2UaYu5CR4rEMWw 9V7fsJjdBqRc9X52phOf5I9xjO7ByiMn81F1z5IFzL4t1TM7xo_NgIEap.uCYSDPRYbUl8SM3Fhl aP0tJtnIdkpyptrg03gu0e1bvgrUrKc2_U0P3fTxsRevzspzuyp.AmF8hw8QKSk3wYvcIBPhuoDl GyFuy8xCog3_amyCJyU0XWXxc1FPnp7MV5G1XFnyU_nL4C1J3.XEBUfDhNIldgED8PuespL_Crwj xGUjPBLq8uANuvsbK5MrLdszkRJfDU4Susa9zGi9nmBoRojDRlCChpAdbn4MwlzfW4nhLeB4ABSJ x1RMoReCTQ2qjmAQwTaBRRPXGSK6MSa1IdJ2w1oMTDEONWjmvWB4jK2V6S0iRMQXM..ljT6ezeDP W5fHlSwpPh0RgnC8AB9uNG0tB6GiPAfTi7Xu1WsPsmQzndfPs6nAbJR380b1XKFOJ88PBGAPQTxn CbxoCQSl5yEEY4N.IbUx99ZAdX1qX7Xn4ep95GvIagyFfgoZddYCHa15.Kmas5sB7jpiQqwLTw3b s1rJmtNcCA2rboXaZDGYklmd6d4QZad4bwlrMLnR5Ga4R6naT2SUyGi1G6wMYaSp5jVtHvolbSCT pQ7TlbFh6CvzHKSEQ39ouMWtYrqx0m6lM3StapJhu3L64n.WwbF0t.6r2Knkbqxe4aczIm.N_YQP AYUoGQsr_6fycvMOvqwpl7eeCqtONBNkV2X9vJZqFVK8cjHRWv686ZusTpyWGreuR5fwoq1d4Jqm 96LVYhHBnoBjiOxKntBsi4tlfxBYYR5SN1Cl_oloV_eX58v8Ib9uOFvOJOH3Yk3i9oRKjzoHFH8c EK3RATqkoqr6EgfY0v9EbFuPPPfz0Zz3DOJMjlhFXBZnNUYvcRlyhCgIm15dFs80KO1LNf4lItQG ex7scJSOc8a17CHZ26spFb77HrlXX9FV8tavizAwm1QXtPe1.tRGDnWPUWMdHipu2RF31Fca5zFT sA_ob1.RckeOuq4A8SoOLySLPayeuam58G2sQQzpUQeGytW9P2yS49ffokvZClE4WSuoC_THV2yC pWdc0PXtoxYihFtBU_YUByUNOhfX_Zb.lyo_BS544vXvVjCDAJyBJ0MmhynQ_1bP.UVAuXFye7SQ K.aZwT.QqBCMpAN_1NbYwbfRz94xr3JxJONSjI1JERZlVv_RZYp5Tn2yZ7MK9q5evJus_ajbw96f BWaX2R_6mzf7l_QvCG.gMq8OlltVcxXhze3L9kR.NDoN3E.PeApj6HJ00JYhYrON1raPWBoIShrH _qaHRYoV2KsZk0IApte8PZrkNMQ.hn4LI3z0iXrRnJQ19r303._yHdydR18FZ1NyZa68ANnJPouu x2lQEUa3nFC5SmsMk.zH_OOr1VA_8DGtzjzahzZTCMvF4j55.6b.pMQnbcu5sy2bCcoWAM1CDHM0 CIrezw98JaySAf9rkFk91bKzlMJXRtsBbva0ZDQRV6qgZ24k.7VICfD6OH322MV5OrDaN9nJDOwq RXMNHtCDtHGCFveS8a9yX1Q66jFlc6Om9Te5pTbNATXDf2EbyvxRLwgpSG7jpnYv7wiQ7.TaXNVJ 0kjbdrecJFuYkn1R3GYpuBnQz6K6QcSIUbmOo9MBrxFKhHj8LQlf_icKeU6YeR.BWXkBBpUL15SS m_tY.hFMSkozITpugGR6t1sHlQjx9tusqgz1klAJfGRxGMg9OqDGmu_kQHfQFbJR5DE2NZKrPhe. fQ5z3tIyEFATYlnDkqhTJcHaHO_mmSnMYn65S8IjltqJqQkRPUCdUkKGSXfmduzOhiBFshjNmIV. Twba8zUfPEuCWtE1qf208kuH1bUCTJdf..Zp7v6VwxKVigvu2Sbw0qTT6NlUtp7M.zul9Mmvv1qA .zEnYpl3OZfDq6Jwx6r.mqFcFZ8_arV_AiBilY0hrCpNQNWMWCZGhS6V2dU_7lgJXT8MuAXE91_H M X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Thu, 28 Apr 2022 06:47:42 +0000 Received: by hermes--canary-production-ne1-75b69fcf97-dzsj8 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 06a0a7e357b85a4c8ea2691b91343db3; Thu, 28 Apr 2022 06:47:39 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: FYI: RPi* firmware tagged 1.20210805 appears to be the last to be bootable by FreeBSD via fdt use; sequence of 2 failure modes after that Date: Wed, 27 Apr 2022 23:47:37 -0700 References: <836019DE-531E-4B49-8A82-0D8F84885C21@yahoo.com> <8291972B-A953-4FD9-AA67-9F1F15AA3940@yahoo.com> To: "freebsd-arm@freebsd.org" In-Reply-To: <8291972B-A953-4FD9-AA67-9F1F15AA3940@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4KpmQR3kNfz4YB3 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=KSAXaDxi; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-1.51 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-0.98)[-0.981]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.206:from]; NEURAL_SPAM_SHORT(0.98)[0.976]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MLMMJ_DEST(0.00)[arm]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-ThisMailContainsUnwantedMimeParts: N [Just an FYI: I got ahold of the RPi3B and discovered that it was not bootable via RPi* firmware tagged 1.20210805 . In fact it barely produced any output on the serial console: very early failure. Reverting to the prior one, 1.20210727, worked for the RPi3B and the RPi4B.] [I've not added to the below and have removed the long text block of RPi4B boot failure output.] On 2022-Apr-24, at 05:36, Mark Millard wrote: > [I may have also found what leads to the extra messages for > the 2nd failure mode, an independent issue it turns out.] >=20 > On 2022-Apr-24, at 04:37, Mark Millard wrote: >=20 >> [I think I found the reason for the boot crash that is >> a common failure to both failure modes. The 2nd mode >> has other issues I've not analyzed.] >>=20 >> On 2022-Apr-23, at 23:45, Mark Millard wrote: >>=20 >>> The following is based on a microsd card with 13.1-RC4 on >>> it were I'd previously substituted my U-Boot 2022.04 build >>> and tested with the RPi* firmware that is in the 13.1-RC4 >>> image. Here I've tried replacing the RPi* firmware and >>> holding the rest constant. >>>=20 >>> The boot tests are on a 8 GiByte RPi4B Rev 1.14 with the >>> B0T stepping. I've not been copying over the linux kernels, >>> which they also bundle with the firmware. >>>=20 >>> [13.1-RC4 is just what I happened to use. I doubt anything >>> here is special to 13.* or stable/13 or main [so: 14]. >>> (I do not use 12.* or stable/12.)] >>>=20 >>> The observed status went like . . . >>>=20 >>>=20 >>> firmware-1.20210805/boot/ >>>=20 >>> The RPi* release tagged 1.20210805 is the last version that >>> FreeBSD booted with. (Other than booting, logging in, and >>> shutting down, I've not been testing other aspects of >>> operation.) >>>=20 >>> =46rom what I've read, firmware-1.20210805/boot/ should be >>> recent enough to handle the Rev 1.15 related PMIC variation. >>>=20 >>> [I'll note that firmware build dates need not be the same day >>> as the date encoded into the tag --in fact it is usually some >>> earlier day. On rare occasion it can be a lot earlier, and >>> there is an example of that below.] >>>=20 >>>=20 >>> After firmware-1.20210805 there are 2 major failure modes. >>> Both stop at the same sort of point in the messaging --but >>> there is a huge difference in the count of earlier error >>> messages. It looks to me like all the issues require >>> FreeBSD changes if modern RPi* firmware/dtb's are to be >>> usable via fdt. >>=20 >> I've noticed a difference between the working context and >> the failing ones (both failure modes). >>=20 >> Failing: >>=20 >> spi0: mem 0x7e204000-0x7e2041ff irq 18 = on simplebus0 >> spibus0: on spi0 >> spibus0: at cs 0 mode 0 >> spibus0: at cs 1 mode 0 >> NOTE BELOW LINES MISSING HERE. >> sdhci_bcm0: mem = 0x7e300000-0x7e3000ff irq 24 on simplebus0 >>=20 >> Working: >>=20 >> spi0: mem 0x7e204000-0x7e2041ff irq 18 = on simplebus0 >> spibus0: on spi0 >> spibus0: at cs 0 mode 0 >> spibus0: at cs 1 mode 0 >> START LINES MISSING ABOVE >> iichb0: mem 0x7e804000-0x7e804fff irq = 26 on simplebus0 >> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 30,31,32,33,34,35,36,37,38,39,40 on simplebus0 >> bcmwd0: mem = 0x7e100000-0x7e100113,0x7e00a000-0x7e00a023,0x7ec11000-0x7ec1101f on = simplebus0 >> bcmrng0: mem 0x7e104000-0x7e104027 on = simplebus0 >> gpioc1: on gpio1 >> END LINES MISSING ABOVE >> sdhci_bcm0: mem = 0x7e300000-0x7e3000ff irq 73 on simplebus0 >>=20 >> In particular: >>=20 >> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 30,31,32,33,34,35,36,37,38,39,40 on simplebus0 >>=20 >> being missing means no bcm_dma_attach and that in turn means >> that the static bcm_dma_sc =3D=3D NULL still. >>=20 >> The panic was: panic: vm_fault failed: ffff000000862134 >>=20 >> where: >>=20 >> ffff000000862134 ldaxr x1, [x9] >>=20 >> which is part of: >>=20 >> int >> bcm_dma_allocate(int req_ch) >> { >> struct bcm_dma_softc *sc =3D bcm_dma_sc; >> int ch =3D BCM_DMA_CH_INVALID; >> int i; >>=20 >> if (req_ch >=3D BCM_DMA_CH_MAX) >> return (BCM_DMA_CH_INVALID); >>=20 >> /* Auto(req_ch < 0) or CH specified */ >> mtx_lock(&sc->sc_mtx); >> . . . >>=20 >> So the likes of &sc->sc_mtx end up being a small offset >> from address zero: >>=20 >> x9: 20 >>=20 >> Thus the panic. >>=20 >> As to how bcm_dma_allocate happened without bcm_dma_attach >> happening first . . . >>=20 >> The working context's dtb has the ordering: >> (I also show mmcnr@ and the brcm,bcm2711-dma >> just for reference.) >>=20 >> dma@7e007000 { >> compatible =3D "brcm,bcm2835-dma"; >> . . . >> mmc@7e300000 { >> compatible =3D "brcm,bcm2835-mmc", = "brcm,bcm2835-sdhci"; >> . . . >> mmcnr@7e300000 { >> compatible =3D "brcm,bcm2835-mmc", = "brcm,bcm2835-sdhci"; >> . . . >> dma@7e007b00 { >> compatible =3D "brcm,bcm2711-dma"; >>=20 >> But the failing context's dtb has the ordering: >> (I also show mmcnr@ and the brcm,bcm2711-dma >> just for reference.) >>=20 >> mmc@7e300000 { >> compatible =3D "brcm,bcm2835-mmc", = "brcm,bcm2835-sdhci"; >> . . . >> dma@7e007000 { >> compatible =3D "brcm,bcm2835-dma"; >> . . . >> mmcnr@7e300000 { >> compatible =3D "brcm,bcm2835-mmc", = "brcm,bcm2835-sdhci"; >> . . . >> dma@7e007b00 { >> compatible =3D "brcm,bcm2711-dma"; >>=20 >> So, for sequential handling in the failing case, the dma@7e007000 >> would use bcm_dma_allocate before the bcm_dma_probe/bcm_dma_attach >> sequence had happened, leading to the crash. >>=20 >> Note: I used "fdt print /" from U-Boot to get the dtb and its >> ordering. This was based on the address that the RPi* firmware >> reports when debugging output is enabled (0x4000 here). >>=20 >>=20 >>> The 1st mode happens for (I've added the -fails notation): >>>=20 >>> firmware-1.20210831-fails/boot/ >>> firmware-1.20210928-fails/boot/ >>> firmware-1.20211007-fails/boot/ >>> firmware-1.20211029-fails/boot/ >>> firmware-1.20211118-fails/boot/ >>> firmware-1.20220308_buster-fails/boot/ >>> (The _buster one has firmware from 2021-Dec-01, which >>> is before all the tagged releases listed below. >>> It looks like the switch to the new major kernel >>> version after buster came with other changes that >>> FreeBSD has not tracked.) >>>=20 >>>=20 >>> The 2nd mode happens for the following. (Again with extra >>> notation.) There are a lot more error messages before the >>> panic happens for these. The firmware builds for these >>> are more recent than for the above list. >>>=20 >>>=20 >>> firmware-1.20220118-fails/boot/ >>>=20 >>> firmware-1.20220120-fails/boot/ >>> firmware-1.20220308-fails-non-kernels-same-as-1.20220120/boot/ >>> (I did not repeat the testing of the unchanged firmware. >>> I just did the "diff -r" to discover the lack of change.) >>>=20 >>> firmware-1.20220328-fails/boot/ >>> = firmware-1.20220331-fails-non-kernels-same-as-firmware-1.20220328-but-for-= bcm2711-dtb-files/boot/ >>> (Since the .dtb for the RPi4B was different, I did test this.) >=20 > It looks like the extra messages, blocks of: >=20 > clk_fixed4: disabled on ofwbus0 > clk_fixed4: Cannot FDT parameters. > device_attach: clk_fixed4 attach returned 6 >=20 > Are tied to new dtb content in 2022's dtb updates: >=20 > cam1_clk { > compatible =3D "fixed-clock"; > #clock-cells =3D <0x00000000>; > status =3D "disabled"; > phandle =3D <0x000000e2>; > }; > . . . > cam0_clk { > compatible =3D "fixed-clock"; > #clock-cells =3D <0x00000000>; > status =3D "disabled"; > phandle =3D <0x000000e4>; > }; >=20 > These 2 did not exist back when the 1st failure mode > started. They appear to be repeatedly processed from > not really being handled --leading to lots of > messages. >=20 > The messages may just be noise for activity that is > not contributing to boot failures at all. So fixing > what I called the 1st failure mode might actually fix > booting for all the firmware versions after the > version tagged 1.20210805 . >=20 >>> The failures look like (each test shown) . . . >>>=20 >>>=20 >>> . . . >>=20 =3D=3D=3D Mark Millard marklmi at yahoo.com