Re: Troubles booting Pi2 from USB using bootcode.bin method

On 2021-Oct-29, at 11:24, bob prohaska <fbsd@www.zefox.net> wrote:
> 
> I should have  stated earlier that the FreeBSD system on
> the USB3 disk orginated from
> FreeBSD-13.0-RELEASE-arm-armv7-GENERICSD.img
> using dd and then repartitioning to add a swap and separate
> /usr partition.
> 
> On Thu, Oct 28, 2021 at 04:53:29PM -0700, Mark Millard wrote:
>> On 2021-Oct-28, at 15:21, Mark Millard <marklmi at yahoo.com> wrote:
>> 
>>> On 2021-Oct-28, at 12:16, bob prohaska <fbsd@www.zefox.net> wrote:
>>> 
>>>> To make a clean start on this thread I've turned on the UART
>>>> for bootcode.bin per Mark's instructions and done a few boot
>>>> attempts with the USB2 and USB3 mechanical disks, singly and
>>>> in unison.
>>>> 
>>>> The bootlogs are in
>>>> http://www.zefox.net/~fbsd/rpi2/bootproblems/
>>>> 
>>>> An immediate curiosity is that on the first try, booting
>>>> with the USB3 device alone worked. I didn't record that
>>>> output, unfortunately.
>>> 
>>> Hmm. Too bad.
>>> 
> Indeed! I thought maybe the failure was fixed by turning on the UART...
> 
>>>> The second attempt failed, as expected,
>>>> and is recorded in bootlog-fail. The third attempt booted both
>>>> USB2 and USB3 disks together, recorded in bootlog.success.
>>> 
>>> The two logs do not have the same set of dtdebug messages
>>> for loading bcm2709-rpi-2-b.dtb . This is long before
>>> u-boot.bin is loaded and so is during the RPi* firmware
>>> time frame not u_Boot or FreeBSD;s loader or FreeBSD's kernel
>>> or FreeBSD's world.
>>> 
>>> From this I infer that there are two different msdosfs's
>>> wtith differing content on the 2 drives and when both
>>> drives are in place .
>>> 
> 
> That's correct. Actually there are three, counting the microSD.

But later you show that start.elf and the like are missing
on the microsd card. That prevents the microsd card from being
involved for the stage getting the errors in the one case
(other than the bootcode.bin content used).

>>> You have not reported on the following for either drive's
>>> msdosfs :
>>> 
>>> # strings ???/start.elf | grep "VC_BUILD_"
>>> 
> 
> The are different. On the USB3 disk, which I want to boot, I get
> bob@www:/boot/msdos % strings start.elf | grep "VC_BUILD_"
> VC_BUILD_ID_USER: dom
> VC_BUILD_ID_TIME: 12:12:09
> VC_BUILD_ID_VARIANT: start
> VC_BUILD_ID_TIME: Feb 25 2021
> VC_BUILD_ID_BRANCH: bcm2711_2
> VC_BUILD_ID_HOSTNAME: buildbot
> VC_BUILD_ID_PLATFORM: raspberrypi_linux
> VC_BUILD_ID_VERSION: 564e5f9b852b23a330b1764bcf0b2d022a20afd0 (clean)

The above matches what I use on the USB3 SSD that I use for
armv7 (Cortex-A7) systems, including the RPi2 v1.1 . But I
do not have access to the RPi2B v1.1 currently to test with.
In my context overlays/* and the rest are from the same
release as my start.elf . But that sort of thing is messier
to check.

> On the USB2 disk, which seems to be needed for booting, I get
> oot@www:/mnt # strings start.elf | grep "VC_BUILD_"
> VC_BUILD_ID_USER: dom
> VC_BUILD_ID_TIME: 16:43:13
> VC_BUILD_ID_BRANCH: master
> VC_BUILD_ID_VARIANT: start
> VC_BUILD_ID_TIME: Nov 19 2019
> VC_BUILD_ID_HOSTNAME: buildbot
> VC_BUILD_ID_PLATFORM: raspberrypi_linux
> VC_BUILD_ID_VERSION: 2354eac70a98807e06bed2149bc0c5613e751c15 (clean)

That is rather old. I've no clue what to expect for it.

>>> Another thing of interest would be something like (both
>>> msdosfs mounts):
>>> 
>>> # diff -rq ... ...
>>> 
>>> in order to see what files have distinctions on the
>>> two media. A diff of the two config.txt files would be
>>> relevant (no -q involvement).

Based on the VC_BUILD_ID_* checks, it looks like
a diff -rq would list a lot or most files as
different, presuming all the files.

> On the USB3 disk config.txt contains
> bob@www:/boot/msdos % more config.txt
> init_uart_clock=3000000
> enable_uart=1
> kernel=u-boot.bin
> kernel7=u-boot.bin
> dtoverlay=mmc
> enable_uart=1
> uart_2ndstage=1
> dtdebug=1

enable_uart=1

is listed twice above.

> On the USB2 disk config.txt contains
> root@www:/mnt # more config.txt
> init_uart_clock=3000000
> enable_uart=1
> kernel=u-boot.bin
> kernel7=u-boot.bin
> dtoverlay=mmc

The above is missing:

uart_2ndstage=1
dtdebug=1

so there likely would be less debug information for
the case when this file is used.

Rerunning tests with both config.txt files having
those llines as well might give additional information.

Swapping ports used on the powered hub for the two drives
might allow controlling which drive's msdosfs file system
is used. (This is not the only port position/swapping that
may be useful for such, it is just an illustration.)

>>>> I'm trying to build u-boot-rpi2 and will try to update the USB3
>>>> disk with it once complete. 
> Still working on that part 8-) 

The failures start before u-boot is involved. This is unlikely
to be sufficient.

>>>> 
>>>> The actual boot sequence using bootcode.bin is still a bit hazy:
>>>> Is it microSD/dos -> USB/dos ->USB/freebsd ? 
>>>> 
>>> 
>>> Based on the log file for success the ordering is
>>> 
>>> bootcode.bin from the microsd card
> 
> So bootcode.bin runs from the microSD, but config.txt and all else
> is picked up from whichever USB disk is recognized first?

Yep: it picks one. Swapping or moving which ports are used
may change which one is used when both are connected.

>>> config.txt (also re-read multiple times later, not listed)
>>> start.elf
>>> fixup.dat
>>> bcm2709-rpi-2-b.dtb
>>> overlays/mmc.dtbo
>>> cmdline.txt (if it exists)
>>> u-boot.bin
>>> efi/boot/bootarm.efi
>>> efi/freebsd/loader.env
>>> /boot/defaults/loader.conf
>>> /boot/device.hints
>>> /boot/loader.conf
>>> /boot/loader.conf.local
>>> /boot/boot/kernel
>>> /boot/kernel/fi.lemon.ko
>>> /boot/kernel/umodem.ko
>>> FreeBSD world

Do both USB* drives also have a bootable
FreeBSD installed?

>>> However the failing one has the following involved
>>> (I omit various lines):
>>> 
>>> . . .
>>> Loading 'bcm2709-rpi-2-b.dtb' to 0x100 size 0x6879
>>> Unknown dtparam 'pwr_led_gpio' - ignored

The above seems odd.

>>> dterror: no symbols found
>>> dtdebug: /__overrides__ node not found
>>> Unknown dtparam 'uart0_clkrate' - ignored
>>> dtdebug: Opened overlay file 'overlays/mmc.dtbo'
>>> brfs: File read: /mfs/sd/overlays/mmc.dtbo
>>> dterror: not a valid FDT - err -9

The above may indicate a corrupted mmc.dtbo . Or it might
be problems reading the drive and so be an example of
garbage-in/garbage-out.

>>> . . .
>>> 
>>> That seqeunce makes no mention of: "using platform 'bcm2835'"
>>> and the like. An example is: "found override pwr_led_gpio".
>>> 
>>> Again, all this looks like tehre are two msdosfs involved and
>>> the two are not the same by content.
>>> 
>> 
>> Another possibility is that you have more in the microsd card's
>> msdosfs than just bootcode.bin so that that microsd card might
>> be the source of alternative files. (That makes up to 3 media
>> that might be sources of files.)
> 
> That's correct. The microSD's msdos partition contains
> -rwxr-xr-x  1 root  wheel  52456 Oct 28 02:12 bootcode.bin
> -rwxr-xr-x  1 root  wheel  52456 Oct 22 16:00 bootcode.bin-e
> -rwxr-xr-x  1 root  wheel  52304 Nov 22  2019 bootcode.old
> -rwxr-xr-x  1 root  wheel      0 Jan 16  2021 timeout
> drwxr-xr-x  1 root  wheel   4096 Jan 16  2021 unused
> 
> I'm expecting that only bootcode.bin and timeout are involved.

For that microsd card msdosfs file system content: yep.



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)