Re: Failed to execute custom kernels which where build on a RPi 4 operated by 13.1-RELEASE

On 2022-Jul-4, at 17:57, Mark Millard <marklmi@yahoo.com> wrote:

> On 2022-Jul-4, at 16:57, Dr. Rolf Jansen <freebsd-rj@cyclaero.com> wrote:
> 
>> Hello!
> 
> Hello.
> 
>> On my brand new RPi 4 B (0xb03115), operated by 13.1-RELEASE, I built 2 custom kernels, with kernel configs from different sources. Building and installing went through without issues.
> 
> Have you tried rebuilding the kernel without customizations to
> see if the installed result boots? (Compare/contrast with having
> the customizations.)
> 
> (I gather that the official 13.1-RELEASE installation does
> boot the modern RPi4B okay. True?)
> 
> What type of boot media are in use in each boot test? microsd card?
> USB3? USB2?
> 
> I have yet to have my hands on a 0xb03115 RPi4B variant (so: Rev 1.5).
> It is my understanding that the RPi4B firmware vintage has to be
> recent enough to correctly handle the new PMIC used on the rev 1.5
> variants:
> 
> QUOTE (of RPi engineers on its forums on 2022-Feb-08):
> The PMIC has been changed. Needs firmware from April 21 or later
> . . .
> The firmware in both Raspberry Pi OS - Buster (legacy) and Bullseye supports this. The bootloader has supported this since Apr 2021 (previous default release).
> END QUOTE
> 
> See: https://forums.raspberrypi.com/viewtopic.php?t=329299
> 
>> cd /
>> fetch https://download.freebsd.org/releases/arm64/aarch64/13.1-RELEASE/src.txz
>> tar -xzf src.txz
>> cd /usr/src
>> 
>> Here is the last kernel config which I used:
>> 
>> cat /usr/src/sys/arm64/conf/GENERIC-RPi4
>> 
>>  include	GENERIC
>>  ident	GENERIC-RPi4
>>  nooptions	SOC_NVIDIA_TEGRA210
>> 
>> 
>> make -j4 buildkernel KERNCONF=GENERIC-RPi4
>> make installkernel KERNCONF=GENERIC-RPi4
>> 
>> When restarting with any of the new kernels, booting stalls after these messages in the serial console:
>> 
>>  ...
>>  sdhci_bcm0-slot0: AC12 err: 0x00000000 | Host ctl2:0x00000000
>>  sdhci_bcm0-slot0: Caps:     0x00000000 | Caps2:    0x00000000
>>  sdhci_bcm0-slot0: Max curr: 0x00000001 | ADMA err: 0x00000000
>>  sdhci_bcm0-slot0: ADMA addr:0x00000000 | Slot int: 0x00000000
>>  sdhci_bcm0-slot0: ===========================================
>>  uhub0: 5 ports with 4 removable, self powered
>>  mmc0: No compatible cards found on bus
>> 
>> The last line is not indicative for the error, since I see this as well with the original GENERIC kernel, only then it does not even think once and continues without pause in the boot sequence.
> 
> Looking at an old, saved capture of the serial console from a
> prior RPi4B boot of main, I see the sequence:
> 
> . . .
> uhub0: 5 ports with 4 removable, self powered
> sdhci_bcm0-slot0: Got command interrupt 0x00030000, but there is no active command.
> sdhci_bcm0-slot0: ============== REGISTER DUMP ==============
> sdhci_bcm0-slot0: Sys addr: 0x00000000 | Version:  0x00009902
> sdhci_bcm0-slot0: Blk size: 0x00000000 | Blk cnt:  0x00000000
> sdhci_bcm0-slot0: Argument: 0x00000000 | Trn mode: 0x00000000
> sdhci_bcm0-slot0: Present:  0x000f0000 | Host ctl: 0x00000001
> sdhci_bcm0-slot0: Power:    0x0000000f | Blk gap:  0x00000000
> sdhci_bcm0-slot0: Wake-up:  0x00000000 | Clock:    0x00003947
> sdhci_bcm0-slot0: Timeout:  0x00000000 | Int stat: 0x00000000
> sdhci_bcm0-slot0: Int enab: 0x01ff00bb | Sig enab: 0x01ff00bb
> sdhci_bcm0-slot0: AC12 err: 0x00000000 | Host ctl2:0x00000000
> sdhci_bcm0-slot0: Caps:     0x00000000 | Caps2:    0x00000000
> sdhci_bcm0-slot0: Max curr: 0x00000001 | ADMA err: 0x00000000
> sdhci_bcm0-slot0: ADMA addr:0x00000000 | Slot int: 0x00000000
> sdhci_bcm0-slot0: ===========================================
> mmc0: No compatible cards found on bus
> mmc1: No compatible cards found on bus
> bcm2835_cpufreq0: ARM 2000MHz, Core 500MHz, SDRAM -1094MHz, Turbo ON
> CPU  0: ARM Cortex-A72 r0p3 affinity:  0
>                   Cache Type = <64 byte D-cacheline,64 byte I-cacheline,PIPT ICache,64 byte ERG,64 byte CWG>
> Instruction Set Attributes 0 = <CRC32>
> Instruction Set Attributes 1 = <>
> . . .
> 
> This boot was via USB3, not the microsd card. Thus the
> 
> mmc1: No compatible cards found on bus
> 
> was expected in my sequence. Were you booting from USB3?
> microsd card? . . .?
> 
> If the official 13.1-RELEASE image copy boots from the same
> type of media, what displays at that point for the official
> build? Is it a "bcm2835_cpufreq0:" line?
> 
> Note: the -1094MHz is from FreeBSD printing unsigned data
> based on a signed interpretation. My configuration is set
> up to run faster than the defaults as well ("ARM 2000MHz").
> 
>> Is anyone able to build !!!working!!! custom kernels with the 13.1-RELEASE sources on the very RPi 4?
> 
> You might want to try a boot -v to have more stages
> output. It might give more accuracy about the last
> stage that can successfully output.
> 

FYI in case it helps . . .

Back in late 2022-Apr I sent notices to the list about
my testing what vintages of RPi* firmware FreeBSD would
tolerate/handle, the end result being:

https://lists.freebsd.org/archives/freebsd-arm/2022-April/001302.html

The pre-Rev 1.5 RPi4B's and the one RPi3B that I had to
test worked for the firmware tagged 1.20210805 but not
for tags after that. (So: after 2021-Apr and so able
to handle the RPi4B Rev 1.5 PMIC.)

Basically, FreeBSD presumes an ordering in the .dtb materials
that is not required and changed in the more recent .dtb .
files. The result for the more recent firmware versions that
fail to mix well with FreeBSD is that FreeBSD tries to do
something with something that is not yet set up for such use
and crashes. The relevant definition that needs to be looked
up to do the setup is later in the modern .dtb content order
but in older .dtb versions happened to be earlier so there
was no obvious failure at the time.

===
Mark Millard
marklmi at yahoo.com