Re: Troubles booting Pi2 from USB using bootcode.bin method

On 2021-Oct-29, at 15:32, Mark Millard <marklmi@yahoo.com> wrote:

> On 2021-Oct-29, at 15:10, Mark Millard <marklmi@yahoo.com> wrote:
> 
>> On 2021-Oct-29, at 11:24, bob prohaska <fbsd@www.zefox.net> wrote:
>>> 
>>> I should have  stated earlier that the FreeBSD system on
>>> the USB3 disk orginated from
>>> FreeBSD-13.0-RELEASE-arm-armv7-GENERICSD.img
>>> using dd and then repartitioning to add a swap and separate
>>> /usr partition.
>>> 
>>> On Thu, Oct 28, 2021 at 04:53:29PM -0700, Mark Millard wrote:
>>>> On 2021-Oct-28, at 15:21, Mark Millard <marklmi at yahoo.com> wrote:
>>>> 
>>>>> On 2021-Oct-28, at 12:16, bob prohaska <fbsd@www.zefox.net> wrote:
>>>>> 
>>>>>> To make a clean start on this thread I've turned on the UART
>>>>>> for bootcode.bin per Mark's instructions and done a few boot
>>>>>> attempts with the USB2 and USB3 mechanical disks, singly and
>>>>>> in unison.
>>>>>> 
>>>>>> The bootlogs are in
>>>>>> http://www.zefox.net/~fbsd/rpi2/bootproblems/
>>>>>> 
>>>>>> An immediate curiosity is that on the first try, booting
>>>>>> with the USB3 device alone worked. I didn't record that
>>>>>> output, unfortunately.
>>>>> 
>>>>> Hmm. Too bad.
>>>>> 
>>> Indeed! I thought maybe the failure was fixed by turning on the UART...
>>> 
>>>>>> The second attempt failed, as expected,
>>>>>> and is recorded in bootlog-fail. The third attempt booted both
>>>>>> USB2 and USB3 disks together, recorded in bootlog.success.
>>>>> 
>>>>> The two logs do not have the same set of dtdebug messages
>>>>> for loading bcm2709-rpi-2-b.dtb . This is long before
>>>>> u-boot.bin is loaded and so is during the RPi* firmware
>>>>> time frame not u_Boot or FreeBSD;s loader or FreeBSD's kernel
>>>>> or FreeBSD's world.
>>>>> 
>>>>> From this I infer that there are two different msdosfs's
>>>>> wtith differing content on the 2 drives and when both
>>>>> drives are in place .
>>>>> 
>>> 
>>> That's correct. Actually there are three, counting the microSD.
>> 
>> But later you show that start.elf and the like are missing
>> on the microsd card. That prevents the microsd card from being
>> involved for the stage getting the errors in the one case
>> (other than the bootcode.bin content used).
>> 
>>>>> You have not reported on the following for either drive's
>>>>> msdosfs :
>>>>> 
>>>>> # strings ???/start.elf | grep "VC_BUILD_"
>>>>> 
>>> 
>>> The are different. On the USB3 disk, which I want to boot, I get
>>> bob@www:/boot/msdos % strings start.elf | grep "VC_BUILD_"
>>> VC_BUILD_ID_USER: dom
>>> VC_BUILD_ID_TIME: 12:12:09
>>> VC_BUILD_ID_VARIANT: start
>>> VC_BUILD_ID_TIME: Feb 25 2021
>>> VC_BUILD_ID_BRANCH: bcm2711_2
>>> VC_BUILD_ID_HOSTNAME: buildbot
>>> VC_BUILD_ID_PLATFORM: raspberrypi_linux
>>> VC_BUILD_ID_VERSION: 564e5f9b852b23a330b1764bcf0b2d022a20afd0 (clean)
>> 
>> The above matches what I use on the USB3 SSD that I use for
>> armv7 (Cortex-A7) systems, including the RPi2 v1.1 . But I
>> do not have access to the RPi2B v1.1 currently to test with.
>> In my context overlays/* and the rest are from the same
>> release as my start.elf . But that sort of thing is messier
>> to check.
>> 
>>> On the USB2 disk, which seems to be needed for booting, I get
>>> oot@www:/mnt # strings start.elf | grep "VC_BUILD_"
>>> VC_BUILD_ID_USER: dom
>>> VC_BUILD_ID_TIME: 16:43:13
>>> VC_BUILD_ID_BRANCH: master
>>> VC_BUILD_ID_VARIANT: start
>>> VC_BUILD_ID_TIME: Nov 19 2019
>>> VC_BUILD_ID_HOSTNAME: buildbot
>>> VC_BUILD_ID_PLATFORM: raspberrypi_linux
>>> VC_BUILD_ID_VERSION: 2354eac70a98807e06bed2149bc0c5613e751c15 (clean)
>> 
>> That is rather old. I've no clue what to expect for it.
>> 
>>>>> Another thing of interest would be something like (both
>>>>> msdosfs mounts):
>>>>> 
>>>>> # diff -rq ... ...
>>>>> 
>>>>> in order to see what files have distinctions on the
>>>>> two media. A diff of the two config.txt files would be
>>>>> relevant (no -q involvement).
>> 
>> Based on the VC_BUILD_ID_* checks, it looks like
>> a diff -rq would list a lot or most files as
>> different, presuming all the files.
>> 
>>> On the USB3 disk config.txt contains
>>> bob@www:/boot/msdos % more config.txt
>>> init_uart_clock=3000000
>>> enable_uart=1
>>> kernel=u-boot.bin
>>> kernel7=u-boot.bin
>>> dtoverlay=mmc
>>> enable_uart=1
>>> uart_2ndstage=1
>>> dtdebug=1
>> 
>> enable_uart=1
>> 
>> is listed twice above.
>> 
>>> On the USB2 disk config.txt contains
>>> root@www:/mnt # more config.txt
>>> init_uart_clock=3000000
>>> enable_uart=1
>>> kernel=u-boot.bin
>>> kernel7=u-boot.bin
>>> dtoverlay=mmc
>> 
>> The above is missing:
>> 
>> uart_2ndstage=1
>> dtdebug=1
> 
> Please make the 2 config.txt files have a different number of bytes
> relative to each other. This would allow us to see which USB drive
> was in use from the serial console output: it reports the size
> of the content of the file and then it would be unique.
> 
> Blank lines or comment lines would be sufficient to make the
> difference.
> 
> The re-run the tests and post the results and indicate the
> config.txt contents of both files.

Actually, while one of your two examples do not have all the
debugging output enabled, the two config.txt file contents
that you list have distinct byte counts: 130 vs. 90, if I
count right.

Both log files report reading 130 bytes for config.txt, neither
reports reading 90 (or so) characters for config.txt .

Thus, it appears that both examples are for reading the
same media's config.txt : the one with 130 characters
(counting end-of-line related ones as well). That leaves
reading the media getting inconsistent results.

The timeout file is not the only control related to
delaying booting some for more reliability. Default
values are shown by:

bootcode_delay=0
boot_delay=1
boot_delay_ms=0

See: https://www.raspberrypi.com/documentation/computers/config_txt.html

Given that config.txt comes from a USB* drive, it is not
as obvious how much these might help with later I/O to
the USB* drive. But, for reference:

bootcode_delay is for how long bootcode.bin waits before trying
to load the start*.elf that would be used. (Units: seconds.)

boot_delay is for show long the start8.elf waits before loading
the kernel ( in this case: before loading u-boot.bin ). (Units:
seconds.)

boot_delay_ms adds milliseconds to boot_delay.

>> so there likely would be less debug information for
>> the case when this file is used.
>> 
>> Rerunning tests with both config.txt files having
>> those llines as well might give additional information.
>> 
>> Swapping ports used on the powered hub for the two drives
>> might allow controlling which drive's msdosfs file system
>> is used. (This is not the only port position/swapping that
>> may be useful for such, it is just an illustration.)
>> 
>>>>>> I'm trying to build u-boot-rpi2 and will try to update the USB3
>>>>>> disk with it once complete. 
>>> Still working on that part 8-) 
>> 
>> The failures start before u-boot is involved. This is unlikely
>> to be sufficient.
>> 
>>>>>> 
>>>>>> The actual boot sequence using bootcode.bin is still a bit hazy:
>>>>>> Is it microSD/dos -> USB/dos ->USB/freebsd ? 
>>>>>> 
>>>>> 
>>>>> Based on the log file for success the ordering is
>>>>> 
>>>>> bootcode.bin from the microsd card
>>> 
>>> So bootcode.bin runs from the microSD, but config.txt and all else
>>> is picked up from whichever USB disk is recognized first?
>> 
>> Yep: it picks one. Swapping or moving which ports are used
>> may change which one is used when both are connected.
>> 
>>>>> config.txt (also re-read multiple times later, not listed)
>>>>> start.elf
>>>>> fixup.dat
>>>>> bcm2709-rpi-2-b.dtb
>>>>> overlays/mmc.dtbo
>>>>> cmdline.txt (if it exists)
>>>>> u-boot.bin
>>>>> efi/boot/bootarm.efi
>>>>> efi/freebsd/loader.env
>>>>> /boot/defaults/loader.conf
>>>>> /boot/device.hints
>>>>> /boot/loader.conf
>>>>> /boot/loader.conf.local
>>>>> /boot/boot/kernel
>>>>> /boot/kernel/fi.lemon.ko
>>>>> /boot/kernel/umodem.ko
>>>>> FreeBSD world
>> 
>> Do both USB* drives also have a bootable
>> FreeBSD installed?
>> 
>>>>> However the failing one has the following involved
>>>>> (I omit various lines):
>>>>> 
>>>>> . . .
>>>>> Loading 'bcm2709-rpi-2-b.dtb' to 0x100 size 0x6879
>>>>> Unknown dtparam 'pwr_led_gpio' - ignored
>> 
>> The above seems odd.
>> 
>>>>> dterror: no symbols found
>>>>> dtdebug: /__overrides__ node not found
>>>>> Unknown dtparam 'uart0_clkrate' - ignored
>>>>> dtdebug: Opened overlay file 'overlays/mmc.dtbo'
>>>>> brfs: File read: /mfs/sd/overlays/mmc.dtbo
>>>>> dterror: not a valid FDT - err -9
>> 
>> The above may indicate a corrupted mmc.dtbo . Or it might
>> be problems reading the drive and so be an example of
>> garbage-in/garbage-out.
>> 
>>>>> . . .
>>>>> 
>>>>> That seqeunce makes no mention of: "using platform 'bcm2835'"
>>>>> and the like. An example is: "found override pwr_led_gpio".
>>>>> 
>>>>> Again, all this looks like tehre are two msdosfs involved and
>>>>> the two are not the same by content.
>>>>> 
>>>> 
>>>> Another possibility is that you have more in the microsd card's
>>>> msdosfs than just bootcode.bin so that that microsd card might
>>>> be the source of alternative files. (That makes up to 3 media
>>>> that might be sources of files.)
>>> 
>>> That's correct. The microSD's msdos partition contains
>>> -rwxr-xr-x  1 root  wheel  52456 Oct 28 02:12 bootcode.bin
>>> -rwxr-xr-x  1 root  wheel  52456 Oct 22 16:00 bootcode.bin-e
>>> -rwxr-xr-x  1 root  wheel  52304 Nov 22  2019 bootcode.old
>>> -rwxr-xr-x  1 root  wheel      0 Jan 16  2021 timeout
>>> drwxr-xr-x  1 root  wheel   4096 Jan 16  2021 unused
>>> 
>>> I'm expecting that only bootcode.bin and timeout are involved.
>> 
>> For that microsd card msdosfs file system content: yep.
>> 
> 





===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)