From nobody Sat Jan 07 08:37:55 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Nptrk534sz2sFR4 for ; Sat, 7 Jan 2023 08:38:14 +0000 (UTC) (envelope-from maciphone2@googlemail.com) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Nptrh71Mdz3qpS for ; Sat, 7 Jan 2023 08:38:12 +0000 (UTC) (envelope-from maciphone2@googlemail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=googlemail.com header.s=20210112 header.b="F3ACMhP/"; spf=pass (mx1.freebsd.org: domain of maciphone2@googlemail.com designates 2a00:1450:4864:20::32b as permitted sender) smtp.mailfrom=maciphone2@googlemail.com; dmarc=pass (policy=quarantine) header.from=googlemail.com Received: by mail-wm1-x32b.google.com with SMTP id fm16-20020a05600c0c1000b003d96fb976efso5099536wmb.3 for ; Sat, 07 Jan 2023 00:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20210112; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=6HG9DngKCIgIgjwi8jK7M/RGXmkTjsc4OKkfNZ9D88Q=; b=F3ACMhP/EiUEEVGtIIculguUjE0wgQb62eVKWTcCH8nhBM6TJPnghp6LOE2z6Yl/Nx WIfXxJ3A7uSF5+tTWzOly2FX+NFD2zruhI2GS+SxldPGohRKsljP6L4VwPnGpRbpqSe3 4NsZpbtoxBAHGOU992qwoXjpv0wniTytSmSDOgdGKn9A4sABYx39bdpzdhEfhJFjhQNV h90nJyBIY/CgcG22a0qZIRXwQfIqw2onRTSmGKqa0s9wC28AdQJ9MFPhpJLcYHdIIqJb I2HMPLwQKRHH7tnyFXADnvpMzzv4IB63yqFxI8KGcBl1/mKMXDnG0Krf4ajFlknAxpvU jaXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6HG9DngKCIgIgjwi8jK7M/RGXmkTjsc4OKkfNZ9D88Q=; b=f1bjOrbC7x9yupGdOLMvfvVbwBSVzLKM6PX59FBDtsEKT5GPy1bvnh6j50c+8tyXoT xMQA5dfmJ40ZVwNgwYQGzW5TLJ9XOY7Lpg6pDgDm0zCW2jWfB0bvu+InJRkwB/5MX7XF QR4kdpR5oEC0/ovNtjN9kp7DIgiitKO56dut2A8/pftqUgFdvfKVzhrT95WJrwsH20MU n0X8J4c0AtAqdLyE59QP41tZcV5TIaWrWvar3JJdrh4ejBvvkafOyB22kO6g/NYYWiVM 5c+4CFc+iftxNVj0LYdsTY95dDshdNaqqfcy/gGTtz7qWD9WYJxFlrTXNFog444tUED9 3Ceg== X-Gm-Message-State: AFqh2koFPgCpfEAwrxQXnTpqHWp77wQtZktxybNsH5+4ukcJVH/p6Qtm kWZZLM7Fd7MWtotE/FPnMZ6wuaq6KOo= X-Google-Smtp-Source: AMrXdXuoAa319g23FY/WxliQLhxltgAOMdkJovY5g0UC14ZUA3zIoCjfa/tnZyfF+j4zOy0PMwzabg== X-Received: by 2002:a05:600c:a4d:b0:3cf:6e78:e2ca with SMTP id c13-20020a05600c0a4d00b003cf6e78e2camr45084749wmq.5.1673080690750; Sat, 07 Jan 2023 00:38:10 -0800 (PST) Received: from smtpclient.apple (dynamic-046-114-028-121.46.114.pool.telefonica.de. [46.114.28.121]) by smtp.googlemail.com with ESMTPSA id p9-20020a05600c358900b003cffd3c3d6csm4913684wmq.12.2023.01.07.00.38.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 07 Jan 2023 00:38:10 -0800 (PST) From: =?utf-8?Q?Klaus_K=C3=BCchemann?= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: How to make FreeBSD's kernel boot a RPi4B with modern RPi* firmware Date: Sat, 7 Jan 2023 09:37:55 +0100 References: <9C037D3F-A440-4708-993D-117F313691BB@yahoo.com> <374EC3E5-4CB4-4336-A8B9-7A9CF6151691@yahoo.com> To: Mark Millard , freebsd-arm@freebsd.org In-Reply-To: Message-Id: <9E9C739E-8308-472A-B797-05A37559DD00@googlemail.com> X-Mailer: Apple Mail (2.3731.300.101.1.3) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[googlemail.com,quarantine]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[googlemail.com:s=20210112]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[googlemail.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::32b:from]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[googlemail.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DKIM_TRACE(0.00)[googlemail.com:+]; FREEMAIL_TO(0.00)[yahoo.com,freebsd.org]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org] X-Rspamd-Queue-Id: 4Nptrh71Mdz3qpS X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Hi Mark, `ve tested your "early_dma=E2=80=9C patch now on the CM4(not tested on = the 4b)=E2=80=A6 true, it can boot the latest firmware but after investigating in = what=E2=80=99s new=20 in bcm2711-rpi-cm4.dtb I guess new features are mainly related to CPU = (L2-) caches features , That=E2=80=99s why we get things like : "clk_fixed4: = disabled on ofwbus0 clk_fixed4: Cannot FDT parameters.=E2=80=9C ... so I think we currently won=E2=80=99t benefit much of the new firmware. I had a little hope that an earlier dma could shine a bit more light on = bugs like this :=20 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D260131 But a tiny little hope can=E2=80=99t of course fix bugs :-) , so I think = we have to focus more on fixing existing bugs and=20 Adding device drivers(of course the ToDo-list is not news but still the = truth;-) Thanks for taking attention the current firmware things, always worth to = take a look into. Regards K. > Am 25.12.2022 um 17:37 schrieb Mark Millard : >=20 > On Dec 24, 2022, at 20:14, Mark Millard wrote: >=20 >> On Dec 24, 2022, at 19:15, Mark Millard wrote: >>=20 >>> I finally ran into EARLY_DRIVER_MODULE, BUS_PASS_RESOURCE, >>> BUS_PASS_ORDER_MIDDLE and the like and they allow being >>> sure that the brcm,bcm2835-dma related setup has been done >>> before any use of it is made, despite the order in the >>> Device Tree: use an earlier pass for brcm,bcm2835-dma >>> related attach. This avoids the kernel crashing during >>> boot. >>>=20 >>> The example context used below has: serial console with >>> USB3 SSD boot media (not requiring a usb_pgood_delay >>> setting), booting a stable/13. The RPI4B is a C0T one (no >>> 3 GiByte limitation, for example: the PCIe wrapper logic >>> has been corrected). >>>=20 >>> stable/13's source code changes ( similarly for >>> releng/13.1 ): >>>=20 >>> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> index cab8639bb607..d8b49acfe332 100644 >>> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> @@ -766,5 +766,6 @@ static driver_t bcm_dma_driver =3D { >>>=20 >>> static devclass_t bcm_dma_devclass; >>>=20 >>> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, bcm_dma_devclass, = 0, 0); >>> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, = bcm_dma_devclass, >>> + 0, 0, BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE); >>> MODULE_VERSION(bcm_dma, 1); >>>=20 >>>=20 >>> For reference, a 13S snapshot with my kernel build replacing >>> the snapshot's kernel, was booted with: >>>=20 >>> # strings /boot/msdos/start4.elf | grep VC_BUILD_ID_ >>> VC_BUILD_ID_USER: dom >>> VC_BUILD_ID_TIME: 11:09:05 >>> VC_BUILD_ID_VARIANT: start >>> VC_BUILD_ID_TIME: Oct 26 2022 >>> VC_BUILD_ID_BRANCH: bcm2711_2 >>> VC_BUILD_ID_HOSTNAME: buildbot >>> VC_BUILD_ID_PLATFORM: raspberrypi_linux >>> VC_BUILD_ID_VERSION: c72ad6b26ff40c91ef776b847436094ee63fabee = (clean) >>>=20 >>> There are new things present that FreeBSD reports >>> but ignores, producing messages like: >>>=20 >>> clk_fixed4: disabled on ofwbus0 >>> clk_fixed4: Cannot FDT parameters. >>> device_attach: clk_fixed4 attach returned 6 >>>=20 >>> over and over during part of the boot. It seems to >>> retry as it goes and thus produce so many messages. >>>=20 >>> There was also: >>>=20 >>> fb0: on simplebus0 >>> fb0: changing fb bpp from 0 to 24 >>> mbox0: mbox response error >>> fb0: bcm2835_mbox_fb_init failed, err=3D5 >>> device_attach: fb0 attach returned 6 >>>=20 >>> genet0 is working. >>>=20 >>> I've not checked if the microsd card slot can be used. >>>=20 >>> I used the normal FreeBSD U-Boot since I was not booting >>> the NVM3 media that requires extra time (usb_pgood_delay >>> would be assigned in my own U-Boot builds). >>>=20 >>> For reference, I used my typical sort of config.txt : >>>=20 >>> # more /boot/msdos/config.txt >>> [all] >>> arm_64bit=3D1 >>> dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don >>> dtoverlay=3Dmmc >>> dtoverlay=3Ddisable-bt >>> device_tree_address=3D0x4000 >>> kernel=3Du-boot.bin >>>=20 >>> [pi4] >>> hdmi_safe=3D1 >>> armstub=3Darmstub8-gic.bin >>>=20 >>> # >>> [all] >>> # >>> # Local addition that avoids USB3 SSD boot failures that look like: >>> # uhub_reattach_port: port ? reset failed, error=3DUSB_ERR_TIMEOUT >>> # uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling = port ? >>> initial_turbo=3D60 >>> # U-Boot that has, for example, a built-in usb_pgood_delay = assignment >>> # for a media specific issue added: >>> #kernel=3Du-boot.bin.2022.10.arm64 >>> # >>> # Local additions: >>> enable_uart=3D1 >>> uart_2ndstage=3D1 >>> dtdebug=3D1 >>> disable_commandline_tags=3D1 >>> disable_overscan=3D1 >>> #gpu_mem_1024=3D32 >>> # >>> #program_usb_boot_mode=3D1 >>> #program_usb_boot_timeout=3D1 >>>=20 >>> # Old RPi3's/RPi2Bv1.2's may ignore [pi4] and the like. >>> # That would make the below inappropriate for such contexts. >>> [pi4] >>> # Locally avoid hdmi_safe's dislay scaling: >>> hdmi_safe=3D0 >>> # >>> armstub=3Darmstub8-gic.bin >>> # >>> # Local additions: >>> over_voltage=3D6 >>> arm_freq=3D2000 >>> sdram_freq_min=3D3200 >>> force_turbo=3D1 >>> # >>> #total_mem=3D1024 >>> #total_mem=3D991 >>> [all] >>>=20 >>> [pi3]=20 >>> armstub=3Darmstub8.bin >>> dtoverlay=3Dpwm >>> audio_pwm_mode=3D0 >>> [all] >>>=20 >>>=20 >>> As for main [so: 14], the devclass_t use is gone, thus >>> the source code changes are: >>>=20 >>> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> index 5f9ecb0b7981..83c4c493a66b 100644 >>> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> @@ -764,5 +764,6 @@ static driver_t bcm_dma_driver =3D { >>> sizeof(struct bcm_dma_softc), >>> }; >>>=20 >>> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0); >>> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0, >>> + BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE); >>> MODULE_VERSION(bcm_dma, 1); >>>=20 >>>=20 >>=20 >> I should note that the above is not likely to be >> the most appropriate in detail. The boot reports: >>=20 >> # dmesg -a | grep bcm_dma0 >> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >> bcm_dma0: cannot allocate interrupt >> device_attach: bcm_dma0 attach returned 6 >> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >> bcm_dma0: cannot allocate interrupt >> device_attach: bcm_dma0 attach returned 6 >> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >> bcm_dma0: cannot allocate interrupt >> device_attach: bcm_dma0 attach returned 6 >> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >> bcm_dma0: cannot allocate interrupt >> device_attach: bcm_dma0 attach returned 6 >> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >>=20 >> where that last (working) one has the message >> context: >>=20 >> gic0: mem = 0x40041000-0x40041fff,0x40042000-0x40043fff,0x40044000-0x40045fff,0x400460= 00-0x40047fff irq 30 on simplebus0 >> gic0: pn 0x2, arch 0x2, rev 0x1, implementer 0x43b irqs 256 >> bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >>=20 >> So something involving BUS_PASS_INTERRUPT or later >> (but before, say, BUS_PASS_SUPPORTDEV) may be more >> appropriate. Possibly: >>=20 >> BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE >>=20 >> (so after gic0). >>=20 >>=20 >=20 > So, I'm now using . . . > (leading whitespace possibly not accurately preserved) >=20 >=20 > stable/13's source code changes are ( similarly for > releng/13.1 ): >=20 > # git -C /usr/13S-src/ diff sys/arm/broadcom/bcm2835/bcm2835_dma.c > diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c > index cab8639bb607..6d521d6dcace 100644 > --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c > +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c > @@ -766,5 +766,6 @@ static driver_t bcm_dma_driver =3D { >=20 > static devclass_t bcm_dma_devclass; >=20 > -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, bcm_dma_devclass, = 0, 0); > +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, = bcm_dma_devclass, > + 0, 0, BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE); > MODULE_VERSION(bcm_dma, 1); >=20 >=20 > main's [so: 14's] source code changes are: >=20 > # git -C /usr/main-src/ diff sys/arm/broadcom/bcm2835/bcm2835_dma.c > diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c > index 5f9ecb0b7981..d901447df1e9 100644 > --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c > +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c > @@ -764,5 +764,6 @@ static driver_t bcm_dma_driver =3D { > sizeof(struct bcm_dma_softc), > }; >=20 > -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0); > +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0, > + BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE); > MODULE_VERSION(bcm_dma, 1); >=20 > =3D=3D=3D > Mark Millard > marklmi at yahoo.com