Re: Failed to execute custom kernels which where build on a RPi 4 operated by 13.1-RELEASE

From: Mark Millard <marklmi_at_yahoo.com>
Date: Wed, 06 Jul 2022 15:29:23 UTC
On 2022-Jul-5, at 22:01, Mark Millard <marklmi@yahoo.com> wrote:

> On 2022-Jul-5, at 08:09, Dr. Rolf Jansen <freebsd-rj@cyclaero.com> wrote:
> 
>>> . . .
>> 
>> That would be the second step. The first step would be that somebody else confirms my finding that building and running a custom kernel on a stock FreeBSD 13.1-RELEASE on RPi 4 does not work out. And actually that was my initial question.
>> 
>> - In case somebody raises her/his hand telling, that this worked flawlessly on their system,
>> then I would have a more in deep look, what might have gone wrong here.
>> 
>> - In case the issue would be confirmed, then I would submit a bug report, and the discussion
>> may continue in a more productive way on bugs.freebsd.org.
> 
> Summary of the later material:
> 
> It would appear that if building any kernels are
> broken, it is specific to some custom kernel(s)
> in question, not to building kernels in general.
> 13.1-RELEASE's install is able to build, install,
> and boot its own generic kernel on a 8GiByte
> RPi4B Rev. 1.4.
> 
> How I got to that conclusion . . .
> (Written earlier.)
> 
> I'm doing (written as I go along):
> 
> Establish a USB3 media from FreeBSD-13.1-RELEASE-arm64-aarch64-RPI.img.xz
> and releases/arm64/13.1-RELEASE/src.txz . Set up some basic configuration.
> 
> (Note: growfs is broken for the large expansion. I used dump and restore
> for the ufs partition from a mounted non-grown dd of the .img file to
> other media. I also copied over the msdosfs partition content. I set
> up to have partition-based swap space as well. I used gpt partitioning.)
> 
> Boot via that media on a 8 GiByte RPi4B Rev. 1.4 .
> Do some live setup to finish things off.
> 
> Then:
> 
> root@13R-ufs:~ # cd /usr/src
> root@13R-ufs:~ # time make -j4 kernel-toolchain #  630.15 real 2302.72 user  94.36 sys
> root@13R-ufs:~ # time make -j4 buildkernel      # 1790.12 real 6488.26 user 526.25 sys
> root@13R-ufs:~ # time make -j4 installkernel    #    8.17 real   14.94 user  12.00 sys
> root@13R-ufs:~ # diff -rq /boot/kernel/ /boot/kernel.old/ #??? Reproducible builds ???
> Files /boot/kernel/kernel and /boot/kernel.old/kernel differ
> Files /boot/kernel/kernel.bin and /boot/kernel.old/kernel.bin differ

Turns out the differences are mostly git-context text vs. not (src.txz):

FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC
vs. just:
GENERIC

a couple of times in each pair of kernel* filed. (The shorter
text has null characters to make the space used in the file
pairs the same.)

There is one block of other byte differences in each pair of
kernel files:

11234201 323 <D3> 346 <E6>
11234202 233 <9B>  54 ,
11234203 305 <C5> 371 <F9>
11234204 216 <8E> 144 d
11234205  30 ^X 147 g
11234206 230 <98> 176 ~
11234207  75 =   6 ^F
11234208 335 <DD> 302 <C2>
11234209 144 d   5 ^E
11234210  73 ; 276 <BE>
11234211 305 <C5> 162 r
11234212 146 f 266 <B6>
11234213  55 - 213 <8B>
11234214  61 1 212 <8A>
11234215   4 ^D  45 %
11234216 235 <9D> 145 e
11234217  20 ^P 133 [
11234218 141 a 231 <99>
11234219 203 <83>  47 '
11234220 211 <89> 216 <8E>

That is it for the differences.

> root@13R-ufs:~ # shutdown -r now
> . . .
> Performing sanity check on sshd configuration.
> Starting sshd.
> Starting cron.
> Starting background file system checks in 60 seconds.
> 
> Wed Jul  6 04:22:01 UTC 
> FreeBSD/arm64 (13R-ufs) (ttyu0)
> 
> login: root
> Password:
> Jul  6 04:23:28 13R-ufs login[1210]: ROOT LOGIN (root) ON ttyu0
> Last login: Wed Jul  6 03:18:32 on ttyu0
> FreeBSD 13.1-RELEASE GENERIC
> 
> Welcome to FreeBSD!
> 
> Release Notes, Errata: https://www.FreeBSD.org/releases/
> Security Advisories:   https://www.FreeBSD.org/security/
> FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
> FreeBSD FAQ:           https://www.FreeBSD.org/faq/
> Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
> FreeBSD Forums:        https://forums.FreeBSD.org/
> 
> Documents installed with the system are in the /usr/local/share/doc/freebsd/
> directory, or can be installed later with:  pkg install en-freebsd-doc
> For other languages, replace "en" with a language code like de or fr.
> 
> Show the version of FreeBSD installed:  freebsd-version ; uname -a
> Please include that output and any error messages when posting questions.
> Introduction to manual pages:  man man
> FreeBSD directory layout:      man hier
> 
> To change this login announcement, see motd(5).
> root@13R-ufs:~ # uname -apKU
> FreeBSD 13R-ufs 13.1-RELEASE FreeBSD 13.1-RELEASE GENERIC arm64 aarch64 1301000 1301000
> root@13R-ufs:~ # freebsd-version -kru
> 13.1-RELEASE
> 13.1-RELEASE
> 13.1-RELEASE
> root@13R-ufs:~ # gpart show -pl
> =>       40  468862048    da0  GPT  (224G)
>         40      32728         - free -  (16M)
>      32768     524288  da0p1  13Refi  (256M)
>     557056   29360128  da0p2  13Rswp14  (14G)
>   29917184    4194304         - free -  (2.0G)
>   34111488   33554432  da0p3  13Rswp16  (16G)
>   67665920  356515840  da0p4  13Rufs  (170G)
>  424181760   44680328         - free -  (21G)
> 
> root@13R-ufs:~ # gpart show -p
> =>       40  468862048    da0  GPT  (224G)
>         40      32728         - free -  (16M)
>      32768     524288  da0p1  efi  (256M)
>     557056   29360128  da0p2  freebsd-swap  (14G)
>   29917184    4194304         - free -  (2.0G)
>   34111488   33554432  da0p3  freebsd-swap  (16G)
>   67665920  356515840  da0p4  freebsd-ufs  (170G)
>  424181760   44680328         - free -  (21G)
> root@13R-ufs:~ # df -m
> Filesystem      1M-blocks Used  Avail Capacity  Mounted on
> /dev/gpt/13Rufs    168604 8159 146956     5%    /
> devfs                   0    0      0   100%    /dev
> /dev/gpt/13Refi       255   25    230    10%    /boot/efi
> root@13R-ufs:~ # ls -Tld /usr/obj/usr/src/arm64.aarch64/sys/*/
> drwxr-xr-x  3 root  wheel  91136 Jul  6 04:17:59 2022 /usr/obj/usr/src/arm64.aarch64/sys/GENERIC/
> root@13R-ufs:~ # 
> 
> The build and install seems to have worked just fine,
> allowing booting and operation.
> 
> 
> Notes:
> 
> The builds were done via being logged in via ssh. The
> serial console causes more time to be taken waiting
> for the build output as it progresses, so I avoid
> it for builds.
> 
> This media will be around for some time to possibly
> do other experiments with if desired. Provide
> explicit instructions if you want a build tried. The
> starting context would be as above but the instructions
> might say to "rm -fr" various things first, if
> appropriate.
> 

Trying GENERIC-MMCCAM also worked just fine.
(but the context is set up as USB3 media based,
not microsd card based):

root@13R-ufs:~ # time make -j4 buildkernel   KERNCONF=GENERIC-MMCCAM # 1794.69 real 6478.07 user 552.92 sys
root@13R-ufs:~ # time make -j4 installkernel KERNCONF=GENERIC-MMCCAM #    8.25 real   15.21 user  11.64 sys
root@13R-ufs:~ # shutdown -r now
. . .
Wed Jul  6 15:17:49 UTC 
FreeBSD/arm64 (13R-ufs) (ttyu0)

login: root
Password:
Jul  6 15:18:35 13R-ufs login[1211]: ROOT LOGIN (root) ON ttyu0
Last login: Wed Jul  6 05:31:00 on ttyu0
FreeBSD 13.1-RELEASE GENERIC-MMCCAM
. . .
To change this login announcement, see motd(5).
root@13R-ufs:~ # uname -apKU
FreeBSD 13R-ufs 13.1-RELEASE FreeBSD 13.1-RELEASE GENERIC-MMCCAM arm64 aarch64 1301000 1301000
root@13R-ufs:~ # 


===
Mark Millard
marklmi at yahoo.com