Re: FreeBSD OS installation stuck after selecting Boot installer
- In reply to: Anantharam, Arjun: "Re: FreeBSD OS installation stuck after selecting Boot installer"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Fri, 09 Aug 2024 19:39:10 UTC
Hi,
Wanted to provide an update and get some more guidance for next steps:
We are now able to attach an NVMe SSD on PCIe root port block for FreeBSD to install on. FreeBSD sees this NVMe Disk but fails to mount the disk with errors below. Can someone please provide guidance on how to fix the Disk mount issue and proceed with installtion?
Really appreciate the help 🙂
______ ____ _____ _____
| ____| | _ \ / ____| __ \
| |___ _ __ ___ ___ | |_) | (___ | | | |
| ___| '__/ _ \/ _ \| _ < \___ \| | | |
| | | | | __/ __/| |_) |____) | |__| |
| | | | | | || | | |
|_| |_| \___|\___||____/|_____/|_____/ ``` `
s` `.....---.......--.``` -/
/---------- Welcome to FreeBSD -----------\ +o .--` /y:` +.
| | yo`:. :o `+-
| 1. Boot Installer [Enter] | y/ -/` -o/
| 2. Boot Single user | .- ::/sy+:.
| 3. Escape to loader prompt | / `-- /
| 4. Reboot | `: :`
| 5. Cons: Serial | `: :`
| | / /
| Options: | .- -.
| 6. Kernel: default/kernel (1 of 1) | -- -.
| 7. Boot Options | `:` `:`
| | .-- `--.
| | .---.....----.
\-----------------------------------------/
Exiting menu!
Type '?' for a list of commands, 'help' for more detailed help.
OK set hw.uart.console="mm:0x2058400000,rs:2"
OK set efi.rt.disabled=1
OK boot
Loading kernel...
/boot/kernel/kernel text=0x2a8 text=0x9db150 text=0x260994 data=0x150cb8 data=0x
0+0x2bc000 0x8+0x1516b0+0x8+0x17a5c2/
Loading configured modules...
can't find '/boot/entropy'
can't find '/etc/hostid'
No valid device tree blob found!
WARNING! Trying to fire up the kernel, but no device tree blob found!
---> BootStateExitBootServicesEvent ()
---> SendMmCommand ()
SendMessageToStandaloneMm ()
- sending MM Communicate msg - CommSize: 36, MessageLength: 12
SetBootStateHandler [52] - CommBuffer: 4400E5F628, ComBufferSize: 12
SetBootStateHandler [82] - final MmiHandlerStatus: Success
- received MM Communicate response - CommSize: 36, MessageLength: 12
WatchdogExitBootServicesEvent ()
---> PxeShutdown ()
---> IccNetResetRing ()
IccNetResetRing [46] Clearing Tx ring - base: 480D000200, size: FFE00
---> PxeStop ()
MmInstallProtocolInterface: 27ABF055-B1B8-4C26-8048-748F37BAA2DF 0
---<<BOOT>>---
Copyright (c) 1992-2023 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC arm64
FreeBSD clang version 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15a9eac9)
VT: init without driver.
module scmi already present!
real memory = 16622747648 (15852 MB)
avail memory = 16178614272 (15429 MB)
Starting CPU 1 (10000)
Starting CPU 2 (20000)
Starting CPU 3 (30000)
Starting CPU 4 (40000)
Starting CPU 5 (50000)
Starting CPU 6 (60000)
Starting CPU 7 (70000)
Starting CPU 8 (80000)
Starting CPU 9 (90000)
Starting CPU 10 (a0000)
Starting CPU 11 (b0000)
Starting CPU 12 (c0000)
Starting CPU 13 (d0000)
Starting CPU 14 (e0000)
Starting CPU 15 (f0000)
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
random: unblocking device.
random: entropy device external interface
kbd0 at kbdmux0
acpi0: <INTEL MEVC100>
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
acpi0: Could not update all GPEs: AE_NOT_CONFIGURED
psci0: <ARM Power State Co-ordination Interface Driver> on acpi0
gic0: <ARM Generic Interrupt Controller v3.0> iomem 0x205a000000-0x205a00ffff,0x205a080000-0x205a27ffff on acpi0
its0: <ARM GIC Interrupt Translation Service> mem 0x205a040000-0x205a05ffff on gic0
its1: <ARM GIC Interrupt Translation Service> mem 0x205a060000-0x205a07ffff on gic0
generic_timer0: <ARM Generic Timer> irq 26,27,28 on acpi0
Timecounter "ARM MPCore Timecounter" frequency 112500000 Hz quality 1000
Event timer "ARM MPCore Eventtimer" frequency 112500000 Hz quality 1000
pmu0: <Performance Monitoring Unit> on acpi0
acpi_ged0: <Generic Event Device> irq 16 on acpi0
acpi_ged0: Raw IRQ 334
acpi_button0: <Power Button> on acpi0
pcib0: <Generic PCI host controller> on acpi0
pci0: <PCI bus> on pcib0
pci0: <network, ethernet> at device 1.0 (no driver attached)
nvme0: <Generic NVMe Device> mem 0x2640000000-0x2641ffffff,0x2647480000-0x264749ffff at device 1.1 on pci0
nvme0: unable to allocate MSI-X
pci0: <unknown> at device 1.2 (no driver attached)
pci0: <encrypt/decrypt, network/computer crypto> at device 1.3 (no driver attached)
pci0: <unknown> at device 1.4 (no driver attached)
pci0: <network, ethernet> at device 1.6 (no driver attached)
pci0: <mass storage, NVM> at device 1.7 (no driver attached)
pcib1: <PCI-PCI bridge> at device 3.0 on pci0
pci1: <PCI bus> on pcib1
pcib2: <PCI-PCI bridge> at device 4.0 on pci0
pci2: <PCI bus> on pcib2
pcib3: <PCI-PCI bridge> at device 5.0 on pci0
pci3: <PCI bus> on pcib3
nvme1: <Generic NVMe Device> mem 0x20100000-0x20103fff at device 0.0 on pci3
pcib4: <PCI-PCI bridge> at device 6.0 on pci0
pci4: <PCI bus> on pcib4
nvme2: <Generic NVMe Device> mem 0x20000000-0x20003fff at device 0.0 on pci4
cpu0: <ACPI CPU> on acpi0
armv8crypto0: <AES-CBC,AES-XTS,AES-GCM>
Timecounters tick every 1.000 msec
usb_needs_explore_all: no devclass
CPU 0: ARM Neoverse-N1 r4p0 affinity: 0 0 0
Cache Type = <64 byte D-cacheline,64 byte I-cacheline,PIPT ICache,64 byte ERG,64 byte CWG,IDC,DIC>
Instruction Set Attributes 0 = <DP,RDM,Atomic,CRC32,SHA2,SHA1,AES+PMULL>
Instruction Set Attributes 1 = <RCPC-8.3,DCPoP>
Instruction Set Attributes 2 = <>
Processor Features 0 = <CSV3,CSV2,RAS,GIC,AdvSIMD+HP,FP+HP,EL3,EL2,EL1,EL0 32>
Processor Features 1 = <PSTATE.SSBS MSR>
Memory Model Features 0 = <TGran4,TGran64,TGran16,SNSMem,BigEnd,16bit ASID,256TB PA>
Memory Model Features 1 = <XNX,PAN+ATS1E1,LO,HPD+TTPBHA,VH,16bit VMID,HAF+DS>
Memory Model Features 2 = <EVT-8.2,32bit CCIDX,48bit VA,IESB,UAO,CnP>
Debug Features 0 = <DoubleLock,SPE,2 CTX BKPTs,4 Watchpoints,6 Breakpoints,PMUv3p1,Debugv8p2>
Debug Features 1 = <>
Auxiliary Features 0 = <>
Auxiliary Features 1 = <>
AArch32 Instruction Set Attributes 5 = <RDM,CRC32,SHA2,SHA1,AES+VMULL,SEVL>
AArch32 Media and VFP Features 0 = <FPRound,FPSqrt,FPDivide,DP VFPv3+v4,SP VFPv3+v4,AdvSIMD>
AArch32 Media and VFP Features 1 = <SIMDFMAC,FPHP Arith,SIMDHP Arith,SIMDSP,SIMDInt,SIMDLS,FPDNaN,FPFtZ>
CPU 1: ARM Neoverse-N1 r4p0 affinity: 1 0 0
CPU 2: ARM Neoverse-N1 r4p0 affinity: 2 0 0
CPU 3: ARM Neoverse-N1 r4p0 affinity: 3 0 0
CPU 4: ARM Neoverse-N1 r4p0 affinity: 4 0 0
CPU 5: ARM Neoverse-N1 r4p0 affinity: 5 0 0
CPU 6: ARM Neoverse-N1 r4p0 affinity: 6 0 0
CPU 7: ARM Neoverse-N1 r4p0 affinity: 7 0 0
CPU 8: ARM Neoverse-N1 r4p0 affinity: 8 0 0
CPU 9: ARM Neoverse-N1 r4p0 affinity: 9 0 0
CPU 10: ARM Neoverse-N1 r4p0 affinity: 10 0 0
CPU 11: ARM Neoverse-N1 r4p0 affinity: 11 0 0
CPU 12: ARM Neoverse-N1 r4p0 affinity: 12 0 0
CPU 13: ARM Neoverse-N1 r4p0 affinity: 13 0 0
CPU 14: ARM Neoverse-N1 r4p0 affinity: 14 0 0
CPU 15: ARM Neoverse-N1 r4p0 affinity: 15 0 0
Release APs...done
Trying to mount root from ufs:/dev/ufs/FreeBSD_Install [ro,noatime]...
nda0 at nvme1 bus 0 scbus0 target 0 lun 1
nda0: <INTEL SSDPF21Q032TB L0310600 PHAL2200001M3P2GGN>
nda0: Serial Number PHAL2200001M3P2GGN
nda0: nvme version 1.4
nda0: 3052360MB (6251233968 512 byte sectors)
TCP_ratelimit: Is now initialized
nda1 at nvme2 bus 0 scbus1 target 0 lun 1
nda1: <INTEL SSDPF21Q032TB L0310600 PHAL220000123P2GGN>
nda1: Serial Number PHAL220000123P2GGN
nda1: nvme version 1.4
nda1: 3052360MB (6251233968 512 byte sectors)
mountroot: waiting for device /dev/ufs/FreeBSD_Install...
Mounting from ufs:/dev/ufs/FreeBSD_Install failed with error 19.
Loader variables:
vfs.root.mountfrom=ufs:/dev/ufs/FreeBSD_Install
vfs.root.mountfrom.options=ro,noatime
Manual root filesystem specification:
<fstype>:<device> [options]
Mount <device> using filesystem <fstype>
and with the specified (optional) option list.
eg. ufs:/dev/da0s1a
zfs:zroot/ROOT/default
cd9660:/dev/cd0 ro
(which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
? List valid disk boot devices
. Yield 1 second (for background tasks)
<empty line> Abort manual input
mountroot>
mountroot> ?
List of GEOM managed disk devices:
diskid/DISK-PHAL220000123P2GGN nda1 diskid/DISK-PHAL2200001M3P2GGN nda0
mountroot> ufs:/dev/diskid/DISK-PHAL220000123P2GGN
Trying to mount root from ufs:/dev/diskid/DISK-PHAL220000123P2GGN []...
Attempted recovery for standard superblock: failed
Attempted extraction of recovery data from standard superblock: failed
Attempt to find boot zone recovery data.
Finding an alternate superblock failed.
Check for only non-critical errors in standard superblock
Failed, superblock has critical errors
Mounting from ufs:/dev/diskid/DISK-PHAL220000123P2GGN failed with error 2; retrying for 10 more seconds
-----------------------------------------------------
-----------------------------------------------------------------------
mountroot> ?
List of GEOM managed disk devices:
diskid/DISK-PHAL220000123P2GGN nda1 diskid/DISK-PHAL2200001M3P2GGN nda0
mountroot> ufs:/dev/diskid/DISK-PHAL2200001M3P2GGN
Trying to mount root from ufs:/dev/diskid/DISK-PHAL2200001M3P2GGN []...
Attempted recovery for standard superblock: failed
Attempted extraction of recovery data from standard superblock: failed
Attempt to find boot zone recovery data.
Finding an alternate superblock failed.
Check for only non-critical errors in standard superblock
Failed, superblock has critical errors
Mounting from ufs:/dev/diskid/DISK-PHAL2200001M3P2GGN failed with error 2; retrying for 10 more seconds
________________________________
From: Anantharam, Arjun <arjun.anantharam@intel.com>
Sent: Friday, July 19, 2024 5:04 PM
To: Andrew Turner <andrew@fubar.geek.nz>
Cc: Warner Losh <imp@bsdimp.com>; Mark Millard <marklmi@yahoo.com>; freebsd-arm@freebsd.org <freebsd-arm@freebsd.org>
Subject: Re: FreeBSD OS installation stuck after selecting Boot installer
Hi,
The location from which we are trying to boot FreeBSD is a VFS or RAMDisk method:
OK lsdev
disk devices:
disk0: 4194304 X 512 blocks
disk0p1: EFI
disk0p2: FreeBSD UFS
http: (unknown)
net devices:
net0:
net1:
net2:
net3:
Looks like FreeBSD does not have access to this RAMDisk and hence unable to boot the image. I was digging through FreeBSD forums and found some relevant details. I guess the first step would be to build FreeBSD to prepare it as RAMDisk image, so from the links below i could gather something like this ?
Some Examples:
https://forums.freebsd.org/threads/how-would-i-load-root-image-into-memory-backed-fs-and-run-the-system-from-that.78021/
https://www.reddit.com/r/freebsd/comments/tzm2xi/run_from_ram/
https://forums.freebsd.org/threads/mount-memory-disk-on-boot.61090/
https://forums.freebsd.org/threads/questions-about-ramdisk-in-freebsd.20345/
mdmfs (https://man.freebsd.org/cgi/man.cgi?query=mdmfs&sektion=8&manpath=freebsd-release-ports#EXAMPLES) and mdconfig(https://man.freebsd.org/cgi/man.cgi?query=mdconfig&sektion=8&manpath=freebsd-release-ports):
1. Prepare RAMDisk image
mdconfig -a -t vnode -s 10g -u 0
newfs /dev/md0
mount /dev/md0 /mnt
dump -0aLf - / | (cd /mnt && restore -rf -)
dd if=/dev/md0 of=/path/to/ramdisk.img bs=1M
2. Edit /boot/loader.conf:
initrd_load="YES"
initrd_type="mfs_root"
initrd_name="/boot/ramdisk.img"
vfs.root.mountfrom="ufs:/dev/md0"
vfs.root.mountfrom.options="ro,noatime"
3. Set Boot Variables in the Bootloader
OK set initrd_load="YES"
OK set initrd_type="mfs_root"
OK set initrd_name="/boot/ramdisk.img"
OK set vfs.root.mountfrom="ufs:/dev/md0"
OK set vfs.root.mountfrom.options="ro,noatime"
OK boot
Can someone please clarify if this is a right approach? Any inputs are appreciated!!
Thanks,
Arjun
________________________________
From: Andrew Turner <andrew@fubar.geek.nz>
Sent: Wednesday, July 10, 2024 11:36 AM
To: Anantharam, Arjun <arjun.anantharam@intel.com>
Cc: Warner Losh <imp@bsdimp.com>; Mark Millard <marklmi@yahoo.com>; freebsd-arm@freebsd.org <freebsd-arm@freebsd.org>
Subject: Re: FreeBSD OS installation stuck after selecting Boot installer
On 10 Jul 2024, at 17:03, Anantharam, Arjun <arjun.anantharam@intel.com> wrote:
Hi,
On most of our hardware, i still run into mount issues. Can anyone provide some guidance on this ?
This iso/img file boot is the only option right now for us to boot FreeBSD(no PXE boot option yet), so any inputs are appreciated!!
The issue appears to be the interface the NVMe provides to the kernel.
found-> vendor=0x8086, dev=0x1457, revid=0x21
domain=0, bus=0, slot=1, func=1
class=01-08-02, hdrtype=0x00, mfdev=1
cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords)
lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
intpin=a, irq=255
powerspec 3 supports D0 D3 current D0
MSI supports 1 message, 64 bit, vector masks
MSI-X supports 257 messages in map 0x18
map[10]: type Prefetchable Memory, range 64, base 0x2220000000, size 25, memory disabled
This looks like the NVMe device that the nvme driver attaches to. The class=01-08-02 indicates it’s an NVMe device using the enterprise nvmhci.
…
found-> vendor=0x8086, dev=0x1458, revid=0x21
domain=0, bus=0, slot=1, func=7
class=01-08-00, hdrtype=0x00, mfdev=1
cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords)
lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
intpin=a, irq=255
powerspec 3 supports D0 D3 current D0
MSI-X supports 257 messages in map 0x18
map[10]: type Prefetchable Memory, range 64, base 0x2224000000, size 25, memory disabled
This is the NVMe device the nvme driver doesn’t attach to. As the class is 01-08-00 the driver doesn’t attach. The problem is the 00 at the end.
I don’t know the difference between the different interfaces so am unsure how difficult it would be to support this device.
Andrew