FreeBSD AMD 64 just hangs during boot

Sat Oct 2 07:18:28 PDT 2004

Dear all.

I've been trying for month to get my server to run FreeBSD 5.x AMD64  
without much luck.
So far I always thought that the culprit was the 3ware 9500-8S card.
The server has the following configuration:
-Tyan K8S Pro (S2882) with one Opteron 2.2Ghz , 1GB ECC RAM.
http://www.tyan.com/products/html/thunderk8spro.html

-Main hard drive is a Seagate ST373307LW (SCSI 73GB, 10Krpm) running on  
the on-board Adaptec AIC7902 Ultra320 SCSI adapter
-3ware 9500-8S card driving 6 Western Digital SATA 73GB, 10Krpm disk.

Here is the history of my attempt.
With FreeBSD 5.2.1 for amd64 , the system would boot just fine but the  
3ware card wouldn't be recognized (which was expected as the twa driver  
wasn't present).
Compiling at the time the 3ware driver would just hang when the twa  
kernel module got loaded.
At that point I gave up and installed FreeBSD 5.2.1 for x86.

After a few weeks, I installed FreeBSD-current when the twa driver got  
into the kernel. x86 target no problem, when amd64 target was installed  
as soon as the card was being recognized it would just die.

Yesterday, I got FreeBSD 5.3 RELENG_5 (5.3 Beta6)... x86 just fine, all  
works,
AMD64 version: hangs just after displaying the message that it's  
mounting the / directory.
Different things I tried:
1)From source
upgrading to FreeBSD 5.3 Beta6 from the source on the FreeBSD x86  
FreeBSD 5.3 doing:
 >make buildworld TARGET_ARCH=amd64
 >make buildkernel TARGET_ARCH=amd64
 >make installkernel TARGET_ARCH=amd64
then reboot in single mode..
Still it locks when mounting /
2)Using the mini-install disk: performing a binary upgrade.
Same as 1) Locks when mounting /

3)At that stage I was convinced it was the 3ware card as it wasn't  
recognized before.
So I removed the 3ware card from the server and was amazed to see that  
it wasn't the 3ware card after all, but it seems to be the Adaptec SCSI  
Ultra320!! When this one was working fine in the past.
I thought of a drivers incompatibility, so in the BIOS I disabled  
*everything*:
-serial port, parallel port, acpi, built-in SATA, gigabit interface,  
etc...
Leaving only the SCSI.
Still no luck hangs when mounting / drive

4)I thought it could be some issues with the Seagate drive as I read on  
the net that some drives had issues when interface was too fast. So  
using the Adaptec BIOS interface I slowed down the SCSI to Ultra160,  
then 33 etc...
Still the same lock when mounting /

So what's next? unfortunately, I can't provide logs when booting the  
amd64 version. But as it looks just like the x86 output.

What's bizarre is that if I boot from the FreeBSD AMD64 CD it boots  
fine and all disks and interfaces are recognized.

So here it is, any advice or recommendation will be welcome, I feel  
very frustrated on this one.
My last plan is to format the main disk and re-install from scratch, so  
there would be no x86 , amd64 mixture.

Thank you in advance
Jean-Yves

Copyright (c) 1992-2004 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD 5.3-BETA6 #1: Sat Oct  2 09:33:48 EST 2004
     root@:/usr/obj/usr/src/sys/CUSTOM_SERVER3
ACPI APIC Table: <A M I  OEMAPIC >
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Opteron(tm) Processor 248 (2190.49-MHz 686-class CPU)
   Origin = "AuthenticAMD"  Id = 0xf5a  Stepping = 10

Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE, 
MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
   AMD Features=0xe0500000<NX,AMIE,LM,DSP,3DNow!>
real memory  = 1073676288 (1023 MB)
avail memory = 1041117184 (992 MB)
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
ioapic1 <Version 1.1> irqs 24-27 on motherboard
ioapic2 <Version 1.1> irqs 28-31 on motherboard
npx0: [FAST]
npx0: <math processor> on motherboard
npx0: INT 16 interface
acpi0: <A M I OEMRSDT> on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x5008-0x500b on acpi0
cpu0: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci3: <ACPI PCI bus> on pcib1
ohci0: <OHCI (generic) USB controller> mem 0xfeafd000-0xfeafdfff irq 19  
at device 0.0 on pci3
ohci0: [GIANT-LOCKED]
usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
ohci1: <OHCI (generic) USB controller> mem 0xfeafe000-0xfeafefff irq 19  
at device 0.1 on pci3
ohci1: [GIANT-LOCKED]
usb1: OHCI version 1.0, legacy support
usb1: SMM does not respond, resetting
usb1: <OHCI (generic) USB controller> on ohci1
usb1: USB revision 1.0
uhub1: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 3 ports with 3 removable, self powered
pci3: <display, VGA> at device 6.0 (no driver attached)
fxp0: <Intel 82551 Pro/100 Ethernet> port 0xac00-0xac3f mem  
0xfeaa0000-0xfeabffff,0xfeafc000-0xfeafcfff irq 18 at device 8.0 on  
pci3
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> on miibus0
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:e0:81:29:c0:30
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <AMD 8111 UDMA133 controller> port  
0xffa0-0xffaf,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 7.1 on pci0
ata0: channel #0 on atapci0
ata1: channel #1 on atapci0
pci0: <serial bus, SMBus> at device 7.2 (no driver attached)
pci0: <bridge, PCI-unknown> at device 7.3 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 10.0 on pci0
pci2: <ACPI PCI bus> on pcib2
ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter> port  
0x8800-0x88ff,0x9000-0x90ff mem 0xfc9fc000-0xfc9fdfff irq 24 at device  
6.0 on pci2
ahd0: [GIANT-LOCKED]
aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
ahd1: <Adaptec AIC7902 Ultra320 SCSI adapter> port  
0x9400-0x94ff,0x9800-0x98ff mem 0xfc9fe000-0xfc9fffff irq 25 at device  
6.1 on pci2
ahd1: [GIANT-LOCKED]
aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
bge0: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2003> mem  
0xfc9a0000-0xfc9affff,0xfc9b0000-0xfc9bffff irq 24 at device 9.0 on  
pci2
miibus1: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> on miibus1
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,  
1000baseTX-FDX, auto
bge0: Ethernet address: 00:e0:81:29:c0:b0
bge1: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2003> mem  
0xfc9d0000-0xfc9dffff,0xfc9e0000-0xfc9effff irq 25 at device 9.1 on  
pci2
miibus2: <MII bus> on bge1
brgphy1: <BCM5704 10/100/1000baseTX PHY> on miibus2
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,  
1000baseTX-FDX, auto
bge1: Ethernet address: 00:e0:81:29:c0:b1
pci0: <base peripheral, interrupt controller> at device 10.1 (no driver  
attached)
pcib3: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci1: <ACPI PCI bus> on pcib3
3ware device driver for 9000 series storage controllers, version:  
2.50.00.000
twa0: <3ware 9000 series Storage Controller> port 0x7800-0x78ff mem  
0xfb800000-0xfbffffff,0xfc6ffc00-0xfc6ffcff irq 29 at device 1.0 on  
pci1
twa0: [GIANT-LOCKED]
twa0: 8 ports, Firmware FE9X 2.02.00.012, BIOS BE9X 2.02.01.037
pci0: <base peripheral, interrupt controller> at device 11.1 (no driver  
attached)
acpi_button0: <Power Button> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse, device ID 3
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0 port 0x3f8-0x3ff irq 4 on acpi0
sio0: type 16550A
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
sio1 port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
fdc0: <floppy drive controller (FDE)> port 0x3f7,0x3f0-0x3f5 irq 6 drq  
2 on acpi0
fdc0: [FAST]
orm0: <ISA Option ROMs> at iomem 0xd1000-0xd27ff,0xc0000-0xc7fff on isa0
pmtimer0 on isa0
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on  
isa0
Timecounter "TSC" frequency 2190488152 Hz quality 800
Timecounters tick every 10.000 msec
acpi_cpu: throttling enabled, 8 steps (100% to 12.5%), currently 100.0%
acd0: CDRW <LITE-ON LTR-52327S/QS57> at ata1-slave UDMA33
Waiting 15 seconds for SCSI devices to settle
(probe16:ahd0:0:0:0): No or incomplete CDB sent to device.
(probe16:ahd0:0:0:0): Protocol violation in Message-in phase.   
Attempting to abort.
(probe16:ahd0:0:0:0): Abort Message Sent
(probe16:ahd0:0:0:0): SCB 15 - Abort Tag Completed.
found == 0x1
da0 at ahd0 bus 0 target 0 lun 0
da0: <SEAGATE ST373307LW 0007> Fixed Direct Access SCSI-3 device
da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged  
Queueing Enabled
da0: 70007MB (143374744 512 byte sectors: 255H 63S/T 8924C)
da1 at twa0 bus 0 target 0 lun 0
da1: <3ware Logical Disk 00 1.00> Fixed Direct Access SCSI-0 device
da1: 100.000MB/s transfers
da1: 333735MB (683489280 512 byte sectors: 255H 63S/T 42545C)
Mounting root from ufs:/dev/da0s1a

*** THIS IS WHERE BOOTING THE AMD64 VERSION WOULD HANG

bge0: gigabit link up

---
Jean-Yves Avenard
Hydrix Pty Ltd - Embedding the net
www.hydrix.com | fax +61 3 9572 2686 | phone +61 3 9572 0686 ext 100
VoIP: direct: 100 at voip.hydrix.com, general: info at voip.hydrix.com