Re: Failed to execute custom kernels which where build on a RPi 4 operated by 13.1-RELEASE

From: John Kennedy <warlock_at_phouka.net>
Date: Tue, 05 Jul 2022 13:51:50 UTC
  Let me take a slightly different track here...

  I believe you've said that 14.0-CURRENT (which works without hanging) and
13.1-STABLE have fixed the RTC problem but you'd specifically like to be
running 13.1-RELEASE.  I think you've implied that you've booted up on
13.1-RELEASE (but did have RTC problem, presumably because the changed
haven't been MFCed all the way into the releng/13.1 branch.  The delay
is the whole reason I'm running stable/13 myself, although like I said
for amd64 reasons and just keeping the arm64 systems par.

  While Mark has made me a bit paranoid with what could go wrong at the
pre-FreeBSD stages, if you can reliably recompile 14, those wouldn't
seem to be an issue, if everything else is constant.

  Personally, I wouldn't run 14 in a production environment.  It's been
very good to me in a bhyve environment, but it's not called -STABLE for
a reason.  The reason I like releng/13.1 over stable/13 is that I have a
Pavlovian urge to keep things patched.  So I can totally appreciate you
wanting to run something stable and secure that doesn't get too many
updates.  I'd just try to aim you at stable/13 vs main, at least if you
can't backport the RTC fix into releng/13.1.

  But that being said, still a custom kernel of sorts, yes?

  So is your 14.x setup the same as your -RELEASE setup?  When I
bootstrapped myself initially, I burned a SDCARD and booted up (passed,
so check).  Then I did the bsdinstall onto my USB disk, stomped on the
EFI/msdos contents with the known-good SDCARD contents.  Yanked out
SDCARD, booted up, ok, USB-only known-good there.

  Happily, I could pkg-install things (like git) to recompile the
kernel+world.  Reboot with that, "custom" kernel check.  Recompiling all
the packages locally was a nice stress-test.

  I don't think "downgrading" from 14 to 13 is generally recommended (in
my case, ZFS options would generally get me, but I believe you said
you're UFS).

  Basically, you've got situations where uboot can hand off to modern
FreeBSD kernels on your hardware.  I think you're seeing a lot of lines
of text because we're being paranoid about blind spots where you can't
see some missing error message that would pinpoint the problem.

  All else being equal, if you can boot a stock 13.1-RELEASE kernel
image, you should be able to recompile that kernel in place and expect
it to boot.  So we're looking for the "not equal", something that we're
assuming that is wrong.