Re: 13.1-RELEASE image fails to boot Rev 1.4 8 GiByte RPi4B (B0T vintage SOC) when dd'd to the example USB3 media then used to try to boot via USB3 port

On 2022-Jul-9, at 18:59, Mark Millard <marklmi@yahoo.com> wrote:

> On 2022-Jul-5, at 15:45, Mark Millard <marklmi@yahoo.com> wrote:
> 
>> On 2022-Jul-5, at 14:17, Mark Millard <marklmi@yahoo.com> wrote:
>> 
>>> Summary for Rev 1.4 8 GiByte RPi4B (B0T vintage SOC) context:
>>> 
>>> A) One type of USB3 media (USB2 compatible) works for booting via USB2-port.
>>> B) The same media fails for USB3-port based boot attempts.
>>> C) I boot the same type of media for main [so: 14] instead of 13.1-RELEASE.
>>> D) An alternate type of USB3 media (USB2 compatible) booted via USB3 just
>>> fine via 13.1-RELEASE.
>>> 
>>> The details . . .
>>> 
>>> I intended for for use in the Rev 1.4 8 GiByte RPi4B's
>>> by first going onto USB3 media (of the same kind I
>>> normally use on the RPi4B's with main and the like).
>>> So, I tried: 
>>> 
>>> # dd if=FreeBSD-13.1-RELEASE-arm64-aarch64-RPI.img of=/dev/da0 \
>>> bs=1m conv=sync status=progress
>>> 
>>> and then tried to boot the RPi4B via the media and it gets
>>> the following (the kernel loads and runs but . . .):
>>> 
>>> . . .
>>> Trying to mount root from ufs:/dev/ufs/rootfs [rw]...
>>> uhub0: 5 ports with 4 removable, self powered
>>> ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0
>>> uhub1 on uhub0
>>> uhub1: <vendor 0x2109 USB2.0 Hub, class 9/0, rev 2.10/4.21, addr 1> on usbus0
>>> Root mount waiting for: usbus0
>>> uhub1: 4 ports with 4 removable, self powered
>>> Root mount waiting for: usbus0
>>> uhub_reattach_port: port 3 reset failed, error=USB_ERR_TIMEOUT
>>> uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 3
>>> mountroot: waiting for device /dev/ufs/rootfs...
>>> Mounting from ufs:/dev/ufs/rootfs failed with error 19.
>>> 
>>> Loader variables:
>>> vfs.root.mountfrom=ufs:/dev/ufs/rootfs
>>> vfs.root.mountfrom.options=rw
>>> 
>>> Manual root filesystem specification:
>>> <fstype>:<device> [options]
>>>    Mount <device> using filesystem <fstype>
>>>    and with the specified (optional) option list.
>>> 
>>>  eg. ufs:/dev/da0s1a
>>>      zfs:zroot/ROOT/default
>>>      cd9660:/dev/cd0 ro
>>>        (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
>>> 
>>> ?               List valid disk boot devices
>>> .               Yield 1 second (for background tasks)
>>> <empty line>    Abort manual input
>>> 
>>> mountroot> 
>>> 
>>> Plugging into the other USB3 port and attempting the
>>> power on boot just changes the port numbers from 3 to 2
>>> in the sequence.
>>> 
>>> Plugging into a USB2 port does not have the problem and
>>> it completes the boot and operates. (The media is USB3
>>> but USB2 compatible as well.)
>>> 
>>> Adding to /boot/loader.conf :
>>> 
>>> # First, 3 additions:
>>> kern.cam.boot_delay=10000
>>> vfs.mountroot.timeout=10
>>> vfs.root_mount_always_wait=1
>>> 
>>> and retrying via a USB3 port just results in:
>>> 
>>> . . .
>>> Root mount waiting for: usbus0 CAM
>>> uhub_reattach_port: port 3 reset failed, error=USB_ERR_TIMEOUT
>>> uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 3
>>> Root mount waiting for: CAM
>>> Root mount waiting for: CAM
>>> Root mount waiting for: CAM
>>> Root mount waiting for: CAM
>>> Root mount waiting for: CAM
>>> Root mount waiting for: CAM
>>> Mounting from ufs:/dev/ufs/rootfs failed with error 2; retrying for 10 more seconds
>>> Mounting from ufs:/dev/ufs/rootfs failed with error 2.
>>> 
>>> Loader variables:
>>> vfs.root.mountfrom=ufs:/dev/ufs/rootfs
>>> vfs.root.mountfrom.options=rw
>>> 
>>> Manual root filesystem specification:
>>> <fstype>:<device> [options]
>>>    Mount <device> using filesystem <fstype>
>>>    and with the specified (optional) option list.
>>> 
>>>  eg. ufs:/dev/da0s1a
>>>      zfs:zroot/ROOT/default
>>>      cd9660:/dev/cd0 ro
>>>        (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
>>> 
>>> ?               List valid disk boot devices
>>> .               Yield 1 second (for background tasks)
>>> <empty line>    Abort manual input
>>> 
>>> mountroot> 
>>> 
>>> So: same problem.
>>> 
>>> For reference, from the media being plugged into a different
>>> aarch64 FeeBSD machine running main:
>>> 
>>> usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device Samsung PSSD T7 Touch (0x04e8:0x4001)
>>> ugen1.5: <Samsung PSSD T7 Touch> at usbus1
>>> umass0 on uhub4
>>> umass0: <Samsung PSSD T7 Touch, class 0/0, rev 3.20/1.00, addr 9> on usbus1
>>> umass0:  SCSI over Bulk-Only; quirks = 0x0100
>>> umass0:6:0: Attached to scbus6
>>> da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
>>> da0: <Samsung PSSD T7 Touch 0> Fixed Direct Access SPC-4 SCSI device
>>> da0: Serial Number REDACTED
>>> da0: 400.000MB/s transfers
>>> da0: 953869MB (1953525168 512 byte sectors)
>>> da0: quirks=0x2<NO_6_BYTE>
>>> 
>>> The RPi4B is of a vintage that has the "3 GiByte DMA" issue
>>> present, not that it would be likely to be contributing here.
>>> 
>>> 
>>> So I then tried the sequence using a different type of USB3
>>> media (also USB2 compatible), placed in a USB3 port to boot:
>>> 
>>> usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device OWC Envoy Pro mini (0x1e91:0xa2a5)
>>> ugen1.5: <OWC Envoy Pro mini> at usbus1
>>> umass0 on uhub4
>>> umass0: <OWC Envoy Pro mini, class 0/0, rev 3.00/1.00, addr 11> on usbus1
>>> umass0:  SCSI over Bulk-Only; quirks = 0x0100
>>> umass0:6:0: Attached to scbus6
>>> da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
>>> da0: <OWC Envoy Pro mini 0> Fixed Direct Access SPC-4 SCSI device
>>> da0: Serial Number REDACTED
>>> da0: 400.000MB/s transfers
>>> da0: 228936MB (468862128 512 byte sectors)
>>> da0: quirks=0x2<NO_6_BYTE>
>>> 
>>> For this media, the sequence worked and booted successfully.
>>> 
>>> 
>>> So the issue is somehow specific to some USB3 media 
>>> used in the USB3 ports but not to others. "reset failed,
>>> error=USB_ERR_TIMEOUT" may need more time or better
>>> recovery if a wider range of boot devices are to be
>>> supported. May be the T7 Touch needs more than the
>>> standard amount of time to reset as a USB3 device or
>>> some such. (I've no clue about the details.)
>> 
>> I substituted a 13-STABLE kernel.txz expansion and got the
>> same result using that kernel. (There are not .img files
>> for this currently.)
> 
> I took a different path of setting up a stable/13 based
> media, based on my own build context but upgrading a
> 13.1-RELEASE starting point. This stable/13 variant boots
> the RPi4B's just fine via the USB3 media that 13.1-RELEASE
> failed to boot with. I'm not sure what the differences
> that matter are.
> 

Well, the kernel.txz was based on:

Thu, 30 Jun 2022
	• git: 70fd40edb869 - stable/13 - debugnet: Fix an error handling bug in the DDB command tokenizer Mark Johnston

via:

Fri, 01 Jul 2022
	• New FreeBSD snapshots available: stable/13 (20220701 70fd40edb86) Glen Barber

while my build was based on:

stable/13-n251684-815db559eccc-dirty
(815db559eccc is from Wed, 06 Jul 2022)

Between those are:

Fri, 01 Jul 2022
	• git: 39cd7aa134df - stable/13 - USB: add quirks to XHCI Bjoern A. Zeeb
	• git: 66754c01ff95 - stable/13 - XHCI: clear warm and port reset Bjoern A. Zeeb

and that last mentions port reset handling explicitly.
It is, at least, suggestive.

> For reference for the working stable/13 context and
> other content involved:
> 
> # strings -a /boot/efi/start4.elf | grep VC_BUILD_ID_ 
> VC_BUILD_ID_USER: dom
> VC_BUILD_ID_TIME: 12:10:40
> VC_BUILD_ID_VARIANT: start
> VC_BUILD_ID_TIME: Feb 25 2021
> VC_BUILD_ID_BRANCH: bcm2711_2
> VC_BUILD_ID_HOSTNAME: buildbot
> VC_BUILD_ID_PLATFORM: raspberrypi_linux
> VC_BUILD_ID_VERSION: 564e5f9b852b23a330b1764bcf0b2d022a20afd0 (clean)
> 
> (/boot/efi/armstub8-gic.bin does not have very useful
> version-string information.)
> 
> # strings -a /boot/efi/u-boot.bin | grep 'U-Boot 20'
> U-Boot 2021.07 (May 12 2022 - 07:00:33 +0000)
> 
> Note: To my knowledge, the RPi* firmware, armstub8-gic.bin
> file, and u-boot.bin file above are all as they were for
> 13.1-RELEASE's image, other that for 13.1-RELEASE I had
> also tried adding /boot/efi/timeout and I've left that
> in place since then and I've added my usual
> /boot/efi/config.txt material (that had also not made a
> difference for 13.1-RELEASE).
> 
> (/boot/efi/EFI/BOOT/bootaa64.efi and
> /boot/efi/EFI/freebsd/loader.efi do not have very useful
> version-string information.)
> 
> # uname -apKU # manually line split
> FreeBSD 13S-ufs 13.1-STABLE FreeBSD 13.1-STABLE #40
> stable/13-n251684-815db559eccc-dirty: Sat Jul  9 14:02:23 PDT 2022
> root@CA72_16Gp_ZFS:/usr/obj/BUILDs/13S-CA72-nodbg-clang/usr/13S-src/arm64.aarch64/sys/GENERIC-NODBG-CA72
> arm64 aarch64 1301505 1301505
> 
> 
> Unfortunately, it has been some time since a
> 
> FreeBSD-13.1-STABLE-arm64-aarch64-RPI-*.img
> 
> (a snapshot) has managed to build and be distributed.
> I'm not sure what the result would be like for such at
> this point.
> 
> Once such is available, I may do an experiment based on
> just a stable/13 snapshot.
> 
>> But using:
>> 
>> FreeBSD-14.0-CURRENT-arm64-aarch64-RPI-20220701-9aa02d5120a-256480.img
>> 
>> does not have the problem.
> 

===
Mark Millard
marklmi at yahoo.com