problem booting from 2nd drive

[Christos Chatzaras]

I use dump to take the backup from another server, then boot with mfsbsd and run:

gpart destroy -F ada0
gpart destroy -F ada1

gpart create -s gpt ada0
gpart add -b 40 -s 472 -t freebsd-boot ada0
gpart add -s 4G -t freebsd-ufs ada0
gpart add -s 64G -t freebsd-swap ada0
gpart add -s 16G -t freebsd-ufs ada0
gpart add -s 64G -t freebsd-ufs ada0
gpart add -s 16G -t freebsd-ufs ada0
gpart add -s 512G -t freebsd-ufs ada0
gpart add -t freebsd-ufs ada0

newfs -S 4096 -f 4096 -b 32768 ada0p2
newfs -S 4096 -f 4096 -b 32768 -U ada0p4
newfs -S 4096 -f 4096 -b 32768 -U ada0p5
newfs -S 4096 -f 4096 -b 32768 -U ada0p6
newfs -S 4096 -f 4096 -b 32768 -U ada0p7
newfs -S 4096 -f 4096 -b 32768 -U ada0p8

mount /dev/ada0p2 /mnt
cd /mnt
mkdir var tmp usr home home2
mount /dev/ada0p4 /mnt/tmp
mount /dev/ada0p5 /mnt/var
mount /dev/ada0p6 /mnt/usr
mount /dev/ada0p7 /mnt/home
mount /dev/ada0p8 /mnt/home2

cd /mnt/home2
fetch http://xxx.xxx.xxxx.xxx/FreeBSD-11.1-p1-20170905.tar.gz
tar xf FreeBSD-11.1-p1-20170905.tar.gz
rm FreeBSD-11.1-p1-20170905.tar.gz
mv FreeBSD-11.1-p1-20170905 dump

cd /mnt/tmp && restore -r -f /mnt/home2/dump/tmp.dump && rm restoresymtable
cd /mnt/usr && restore -r -f /mnt/home2/dump/usr.dump && rm restoresymtable
cd /mnt/var && restore -r -f /mnt/home2/dump/var.dump && rm restoresymtable
cd /mnt/home && restore -r -f /mnt/home2/dump/home.dump && rm restoresymtable
cd /mnt && restore -r -f /mnt/home2/dump/root.dump && rm restoresymtable
cd

gpart bootcode -b /mnt/boot/pmbr -p /mnt/boot/gptboot -i 1 ada0

umount /mnt/tmp
umount /mnt/var
umount /mnt/usr
umount /mnt/home
umount /mnt/home2
umount /mnt

gmirror label -vb load gm0p2 /dev/ada0p2
gmirror label -vb load gm0p3 /dev/ada0p3
gmirror label -vb load gm0p4 /dev/ada0p4
gmirror label -vb load gm0p5 /dev/ada0p5
gmirror label -vb load gm0p6 /dev/ada0p6
gmirror label -vb load gm0p7 /dev/ada0p7
gmirror label -vb load gm0p8 /dev/ada0p8

shutdown -r now

----

Then the server come back online and run:

sysctl kern.geom.debugflags=17
gpart backup ada0 | gpart restore -F ada1

gmirror insert gm0p2 /dev/ada1p2
gmirror insert gm0p3 /dev/ada1p3
gmirror insert gm0p4 /dev/ada1p4
gmirror insert gm0p5 /dev/ada1p5
gmirror insert gm0p6 /dev/ada1p6
gmirror insert gm0p7 /dev/ada1p7
gmirror insert gm0p8 /dev/ada1p8

gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada1
gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0

shutdown -r now

-----

ada0 and ada1 are the same models and have the same bootcode.

BIOS has default settings, booting from UEFI, then Network boot, then USB disk, then hard disks.

If I don't run "gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada1" and the disk boot priority is ada1 first and then ada0 it boots. If I write the bootcode to ada1 then the problem exist.