Re: Boot from USB on RPi4 8GB?

On 2021-May-29, at 22:17, William Carson <freebsd@dsllsn.net> wrote:

> Hello Mark, sorry to unicast you but I keep getting rejected from freebsd-arm@. I was hoping you could give me some guidance, as it seems you've made quite a bit of progress in this area.
> 
> Is there any documentation or a howto in order to get FreeBSD to boot from a USB3 disk on the RPi4 8GB? I dd'd the latest 13.0-RELEASE image (FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img) to the USB3 SSD, but it does not boot. (The same image works just fine when written to the sdcard.) It looks like it's unable to locate the USB disk and then gets stuck in a loop trying for network boot. When I get to U-Boot prompt, it says there are no USB storage devices. If I issue "usb start", there are still no devices.

You report:

> If I try "usb reset", it seems to find it, but then it displays an error ("scanning bus xhci_pci for devices... Setup ERROR address device command for slot 1. USB device not accepting new address (error=80000000)" and then locks up. I checked that I have the latest bootloader:

(I assume no microsd card was in the microsd card slot.)

I've not seen or heard of that U-Boot report before.
But you made it to U-Boot so the RPi4B firmware
worked for getting U-Boot from the USB3 drive.

This suggests the U-Boot stage is having the problem.

So I've no specific solutions, not having seen the
kind of report before. I've just more generic notes
and questions.

[I'll note that I've had no problem with USB3 SSD
booting the RPi4B's that I have access to (no
microsd card involved).]

One test is to use the microsd card that boots
via the microsd card slot and instead put it in
a USB3 microsd card reader, plug the reader into
a USB3 port, and try to boot from just that.

If that boots, then there would seem to be some
device incompatibility with your specific USB3
"boot" drive. If, instead, booting via the reader
fails, then things are rather odd. (See later
notes about SOC vintages.)

> # rpi-eeprom-update
> BOOTLOADER: up to date
>  CURRENT: Thu 29 Apr 16:11:25 UTC 2021 (1619712685)
>   LATEST: Thu 29 Apr 16:11:25 UTC 2021 (1619712685)
>  RELEASE: stable (/lib/firmware/raspberrypi/bootloader/stable)
>           Use raspi-config to change the release.
> 
> VL805_FW: Using bootloader EEPROM
>    VL805: up to date
>  CURRENT: 000138a1
>   LATEST: 000138a1

Since you got to U-Boot the RPI4B's bootloader worked.
The above is what I currently use in the RPi4B's,
8 GiBYte and 4 GiByte ones.

> If I dd Raspberry Pi OS (2021-05-07-raspios-buster-armhf-lite.img) to the same disk, it boots up no problem. I'm thinking this is a U-Boot/rpi-firmware problem, but I don't really know where to begin.

FYI: if you ever want to use both a 64-bit kernel and
a 64-bit user space, there are BETA's of such (Debian
Buster based):

https://downloads.raspberrypi.org/raspios_lite_arm64/images/
https://downloads.raspberrypi.org/raspios_arm64/images/

(I'm not claiming any gain from such for the problem at hand.)

> I tried building my own image using crochet,

https://wiki.freebsd.org/arm/ reports about crochet:

"Alternatively, images for many boards can be built by crochet (deprecated) or using FreeBSD's release build infrastructure."

There were no commits at https://github.com/freebsd/crochet
between 2019-Oct-06 and 2021-Apr-23. But there are some
on 2021-Apr-24.

I do my own builds based on using make commands but I do
not use the release scripts for building. I build for a
variety of systems.

> but that didn't work either.

> I'm also not sure whether I should be using sysutils/u-boot-rpi4 (I used this) or sysutils/u-boot-rpi-arm64?

Either should generally work. (But see later notes
about RPi4B SOC vintages.)

FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img was based on
sysutils/u-boot-rpi-arm64 . I've historically built and
used a variant of sysutils/u-boot-rpi4 but my history
goes back before sysutils/u-boot-rpi-arm64 existed. I
do not do anything were the configuration variations
involved matter so I'm not familiar with the distinctions.

I've used both a previous 2020.10 based U-Boot and a
more recent 2021.04 based U-Boot.

> After installing sysutils/rpi-firmware (1.20210303.g20210303), should I just copy /usr/local/share/rpi-firmware/* to the boot partition and rename config_rpi4.txt to config.txt?

I assume you mean the msdos file system partition when
you reference "boot partition". The msdos file system
needs to contain:

A) copies of /usr/local/share/rpi-firmware/* materials,
   using config_rpi4.txt content as the config.txt
   content. The copy should be recursive, to pick up
   the likes of the overlay/ subdirectory tree.

B) a copy of an appropriate u-boot.bin  ( from
   /usr/local/share/u-boot/u-boot-rpi-arm64/ or
   /usr/local/share/u-boot/u-boot-rpi4/ ).
   (See later notes as well.)

C) A copy of /boot/loader.efi (the FreeBSD loader)
   placed at/as efi/boot/bootaa64.efi in teh msdos
   file system.

I use a USB3 SSD that has small enough power requirements
to not require a powered hub. (I also use a 5.1V 3.5A
power supply as part of that context.) I've never tried
spinning rust or higher powered USB3 media.

It is unclear what the dd command details were like for
the transfer to the USB3 media. So I just assume that
it was okay.

You may want to have an empty file named timeout in
the msdos file system. It allows for extra time.
(I doubt it helps, since U-Boot did load and start.)

What does:

# rpi-eeprom-config 
[all]
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0
DHCP_TIMEOUT=45000
DHCP_REQ_TIMEOUT=4000
TFTP_FILE_TIMEOUT=30000
ENABLE_SELF_UPDATE=1
DISABLE_HDMI=0
BOOT_ORDER=0xf41

report in your context? (An equivalent command is
"vcgencmd bootloader_config". I show example output
above.)

Can you see the top of the SOC? Or is there a
heatsink or some such on it? (There is something
there to read if it can be seen.)

One possibility is that you have newer hardware that
the normal U-Boot vintages that FreeBSD has used do
not handle.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255080
is about someone that instead of having only:

QUOTE
Old Pi has the following identifying marks on the SOC package:
BROADCOM
2711ZPKFSB06BOT
TE1919
045-23 B3 W
END QUOTE

the person also had an example of a 2 GiByte:

QUOTE
New Pi has the following identifying marks on the SOC package:
BROADCOM
2711ZPKFSB06C0T
TA2105
054-05 B3 W
END QUOTE

The:

2711ZPKFSB06BOT
vs.
2711ZPKFSB06C0T

is significant. If you have a 8GiByte "C0T" RPi4B it
would be the first known example. In such a case, you
might want to try the u-boot referenced in Comment 15 of
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255080 .
It got the 2 GiByte "C0T" to boot. (But that context
could not even boot via the microsd card slot with the
normal media content from 13.0-RELEASE .)

I do not know if the 2021.04 U-Boot that the since
updated port now provides would work for this context
or not. I've no access to any "C0T" RPi4B's (or any
Pi400's or CM4's).



There is a category of USB device that U-Boot still does
not support as far as I know: those where the one device
has multiple storage LUNs (instead of just one Logical
Unit Number identifying storage).

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253983
is about such a context, where we eventially figured out
the "multiple storage LUN" issue.

But the error messages from the U-Boot of the time was
not what you report.


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)