13.1-RELEASE image fails to boot Rev 1.4 8 GiByte RPi4B (B0T vintage SOC) when dd'd to the example USB3 media then used to try to boot via USB3 port

Summary for Rev 1.4 8 GiByte RPi4B (B0T vintage SOC) context:

A) One type of USB3 media (USB2 compatible) works for booting via USB2-port.
B) The same media fails for USB3-port based boot attempts.
C) I boot the same type of media for main [so: 14] instead of 13.1-RELEASE.
D) An alternate type of USB3 media (USB2 compatible) booted via USB3 just
  fine via 13.1-RELEASE.

The details . . .

I intended for for use in the Rev 1.4 8 GiByte RPi4B's
by first going onto USB3 media (of the same kind I
normally use on the RPi4B's with main and the like).
So, I tried: 

# dd if=FreeBSD-13.1-RELEASE-arm64-aarch64-RPI.img of=/dev/da0 \
bs=1m conv=sync status=progress

and then tried to boot the RPi4B via the media and it gets
the following (the kernel loads and runs but . . .):

. . .
Trying to mount root from ufs:/dev/ufs/rootfs [rw]...
uhub0: 5 ports with 4 removable, self powered
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0
uhub1 on uhub0
uhub1: <vendor 0x2109 USB2.0 Hub, class 9/0, rev 2.10/4.21, addr 1> on usbus0
Root mount waiting for: usbus0
uhub1: 4 ports with 4 removable, self powered
Root mount waiting for: usbus0
uhub_reattach_port: port 3 reset failed, error=USB_ERR_TIMEOUT
uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 3
mountroot: waiting for device /dev/ufs/rootfs...
Mounting from ufs:/dev/ufs/rootfs failed with error 19.

Loader variables:
  vfs.root.mountfrom=ufs:/dev/ufs/rootfs
  vfs.root.mountfrom.options=rw

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:zroot/ROOT/default
        cd9660:/dev/cd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot> 

Plugging into the other USB3 port and attempting the
power on boot just changes the port numbers from 3 to 2
in the sequence.

Plugging into a USB2 port does not have the problem and
it completes the boot and operates. (The media is USB3
but USB2 compatible as well.)

Adding to /boot/loader.conf :

# First, 3 additions:
kern.cam.boot_delay=10000
vfs.mountroot.timeout=10
vfs.root_mount_always_wait=1

and retrying via a USB3 port just results in:

. . .
Root mount waiting for: usbus0 CAM
uhub_reattach_port: port 3 reset failed, error=USB_ERR_TIMEOUT
uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 3
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Mounting from ufs:/dev/ufs/rootfs failed with error 2; retrying for 10 more seconds
Mounting from ufs:/dev/ufs/rootfs failed with error 2.

Loader variables:
  vfs.root.mountfrom=ufs:/dev/ufs/rootfs
  vfs.root.mountfrom.options=rw

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:zroot/ROOT/default
        cd9660:/dev/cd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot> 

So: same problem.

For reference, from the media being plugged into a different
aarch64 FeeBSD machine running main:

usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device Samsung PSSD T7 Touch (0x04e8:0x4001)
ugen1.5: <Samsung PSSD T7 Touch> at usbus1
umass0 on uhub4
umass0: <Samsung PSSD T7 Touch, class 0/0, rev 3.20/1.00, addr 9> on usbus1
umass0:  SCSI over Bulk-Only; quirks = 0x0100
umass0:6:0: Attached to scbus6
da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
da0: <Samsung PSSD T7 Touch 0> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number REDACTED
da0: 400.000MB/s transfers
da0: 953869MB (1953525168 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>

The RPi4B is of a vintage that has the "3 GiByte DMA" issue
present, not that it would be likely to be contributing here.


So I then tried the sequence using a different type of USB3
media (also USB2 compatible), placed in a USB3 port to boot:

usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device OWC Envoy Pro mini (0x1e91:0xa2a5)
ugen1.5: <OWC Envoy Pro mini> at usbus1
umass0 on uhub4
umass0: <OWC Envoy Pro mini, class 0/0, rev 3.00/1.00, addr 11> on usbus1
umass0:  SCSI over Bulk-Only; quirks = 0x0100
umass0:6:0: Attached to scbus6
da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
da0: <OWC Envoy Pro mini 0> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number REDACTED
da0: 400.000MB/s transfers
da0: 228936MB (468862128 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>

For this media, the sequence worked and booted successfully.


So the issue is somehow specific to some USB3 media 
used in the USB3 ports but not to others. "reset failed,
error=USB_ERR_TIMEOUT" may need more time or better
recovery if a wider range of boot devices are to be
supported. May be the T7 Touch needs more than the
standard amount of time to reset as a USB3 device or
some such. (I've no clue about the details.)


===
Mark Millard
marklmi at yahoo.com