Random Lockup with FreeBSD 10.2 on SuperMicro Boards

Mon Nov 16 07:13:36 UTC 2015

Hi,

I'm running quiet a few FreeBSD servers on SuperMicro Boards. I'm in the
process of upgrading from 10.1 to 10.2. On the machines running 10.2 I'm
experiencing so random lockups.

The server running fine bit sometimes (about 2-3 month apart the /var
filesystem just locks. Other filesystems on the same drive (mirror-raid)
still working, only when accessing anything on /var blocks the process.

The same machines running with 10.1 don't have this Problem.

All Filesystems are UFS journaled soft-updates.

There is no message on the console or in the log files (as expected as
the log files are on /var)

Here is the output from mount.

/dev/raid/r0p3 on / (ufs, local)
devfs on /dev (devfs, local, multilabel)
/dev/raid/r0p4 on /var (ufs, local, journaled soft-updates)
/dev/raid/r0p5 on /usr (ufs, local, journaled soft-updates)
/dev/raid/r0p6 on /data (ufs, local, journaled soft-updates)
fdescfs on /dev/fd (fdescfs)
procfs on /proc (procfs, local)
/dev/md0 on /tmp (ufs, local)

I've updated so far three server to 10.2. Two of them by using
freebsd-update from 10.1 and one was fresh installed. All of them failed
once or twice since update to 10.2 and never before (running 10.1).

I've attached the dmesg.boot from the last server to fail (fresh installed)

Regards
   Estartu

-- 
-------------------------------------------------
Gerhard Schmidt       | E-Mail: schmidt at ze.tum.de
TU-München	      | Jabber: estartu at ze.tum.de
WWW & Online Services |
Tel: 089/289-25270    |
Fax: 089/289-25257    | PGP-Publickey auf Anfrage

-------------- next part --------------
Copyright (c) 1992-2015 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.2-RELEASE-p7 #0: Mon Nov  2 14:19:39 UTC 2015
    root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz (3392.22-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x306c3  Family=0x6  Model=0x3c  Stepping=3
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,<b11>,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Structured Extended Features=0x2fbb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 34376515584 (32784 MB)
avail memory = 33266585600 (31725 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM SMCI--MB>
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
ioapic0 <Version 2.0> irqs 0-23 on motherboard
random: <Software, Yarrow> initialized
module_register_init: MOD_LOAD (vesa, 0xffffffff80db8e60, 0) error 19
kbd1 at kbdmux0
acpi0: <SUPERM SMCI--MB> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
igb0: <Intel(R) PRO/1000 Network Connection version - 2.4.0> port 0xe020-0xe03f mem 0xf7180000-0xf71fffff,0xf7284000-0xf7287fff irq 16 at device 0.0 on pci1
igb0: Using MSIX interrupts with 9 vectors
igb0: Ethernet address: 00:25:90:c3:a5:3c
igb0: Bound queue 0 to cpu 0
igb0: Bound queue 1 to cpu 1
igb0: Bound queue 2 to cpu 2
igb0: Bound queue 3 to cpu 3
igb0: Bound queue 4 to cpu 4
igb0: Bound queue 5 to cpu 5
igb0: Bound queue 6 to cpu 6
igb0: Bound queue 7 to cpu 7
igb1: <Intel(R) PRO/1000 Network Connection version - 2.4.0> port 0xe000-0xe01f mem 0xf7100000-0xf717ffff,0xf7280000-0xf7283fff irq 17 at device 0.1 on pci1
igb1: Using MSIX interrupts with 9 vectors
igb1: Ethernet address: 00:25:90:c3:a5:3d
igb1: Bound queue 0 to cpu 0
igb1: Bound queue 1 to cpu 1
igb1: Bound queue 2 to cpu 2
igb1: Bound queue 3 to cpu 3
igb1: Bound queue 4 to cpu 4
igb1: Bound queue 5 to cpu 5
igb1: Bound queue 6 to cpu 6
igb1: Bound queue 7 to cpu 7
pci0: <simple comms> at device 22.0 (no driver attached)
pci0: <simple comms> at device 22.1 (no driver attached)
ehci0: <Intel Lynx Point USB 2.0 controller USB-B> mem 0xf7304000-0xf73043ff irq 16 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 0.0 on pci2
pci3: <ACPI PCI bus> on pcib3
vgapci0: <VGA-compatible display> port 0xd000-0xd07f mem 0xf6000000-0xf6ffffff,0xf7000000-0xf701ffff irq 16 at device 0.0 on pci3
vgapci0: Boot video device
ehci1: <Intel Lynx Point USB 2.0 controller USB-A> mem 0xf7303000-0xf73033ff irq 23 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Patsburg (RAID) AHCI SATA controller> port 0xf050-0xf057,0xf040-0xf043,0xf030-0xf037,0xf020-0xf023,0xf000-0xf01f mem 0xf7302000-0xf73027ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
acpi_button0: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart2: <16550 or compatible> port 0x3e8-0x3ef irq 7 on acpi0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xd0800-0xd17ff,0xd1800-0xd27ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: CGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3d0-0x3db iomem 0xb8000-0xbffff on isa0
ppc0: cannot reserve I/O port range
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est4: <Enhanced SpeedStep Frequency Control> on cpu4
est5: <Enhanced SpeedStep Frequency Control> on cpu5
est6: <Enhanced SpeedStep Frequency Control> on cpu6
est7: <Enhanced SpeedStep Frequency Control> on cpu7
random: unblocking device.
usbus0: 480Mbps High Speed USB v2.0
Timecounters tick every 1.000 msec
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ses0 at ahciem0 bus 0 scbus6 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <ST1000NM0033-9ZM173 SN03> ACS-2 ATA SATA 3.x device
ada0: Serial Number xxxxxxxx
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
ada1: <ST1000NM0033-9ZM173 SN03> ACS-2 ATA SATA 3.x device
ada1: Serial Number xxxxxxxx
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad6
GEOM_RAID: Intel-fbcfc6e1: Array Intel-fbcfc6e1 created.
GEOM_RAID: Intel-fbcfc6e1: Disk ada0 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-fbcfc6e1: Subdisk gm0:0-ada0 state changed from NONE to STALE.
GEOM_RAID: Intel-fbcfc6e1: Disk ada1 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-fbcfc6e1: Subdisk gm0:1-ada1 state changed from NONE to STALE.
GEOM_RAID: Intel-fbcfc6e1: Array started.
GEOM_RAID: Intel-fbcfc6e1: Subdisk gm0:0-ada0 state changed from STALE to ACTIVE.
GEOM_RAID: Intel-fbcfc6e1: Subdisk gm0:1-ada1 state changed from STALE to RESYNC.
GEOM_RAID: Intel-fbcfc6e1: Subdisk gm0:1-ada1 rebuild start at 0.
GEOM_RAID: Intel-fbcfc6e1: Volume gm0 state changed from STARTING to SUBOPTIMAL.
GEOM_RAID: Intel-fbcfc6e1: Provider raid/r0 for volume gm0 created.
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #6 Launched!
Timecounter "TSC-low" frequency 1696109722 Hz quality 1000
uhub1: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
Root mount waiting for: usbus1 usbus0
ugen1.2: <vendor 0x8087> at usbus1
uhub2: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.05, addr 2> on usbus1
ugen0.2: <vendor 0x8087> at usbus0
uhub3: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.05, addr 2> on usbus0
Root mount waiting for: usbus1 usbus0
uhub2: 6 ports with 6 removable, self powered
uhub3: 6 ports with 6 removable, self powered
ugen0.3: <vendor 0x0000> at usbus0
uhub4: <vendor 0x0000 product 0x0001, class 9/0, rev 2.00/0.00, addr 3> on usbus0
uhub4: 4 ports with 3 removable, self powered
Root mount waiting for: usbus0
ugen0.4: <vendor 0x0557> at usbus0
ukbd0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 4> on usbus0
kbd0 at ukbd0
Trying to mount root from ufs:/dev/raid/r0p3 [rw]...
WARNING: / was not properly dismounted
ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 4> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0