Inconsistent softupdate after installation of MegaMonitor

[Shuji Kono]

I'm running 2 servers with 4.8-STABLE and MegaRAID Express500.
Recently I installed MegaMonitor1.02 and MegaMgr6.00 but some
inconsistencies on the filesystem are reported since then.

I'm not sure if this was caused by MegaMonitor but I've never
experienced this before using MegaMonitor. Does anyone have ideas?

Both servers have the same hardware specs: Supermicro Superserver 6010H,
PentiumIII 1.0BGHz x 2, 256MB DIMM x 2, Express500, Seagate ST336607LC x
2 (RAID-1)

Server 1:

Postfix core dumped few minutes after installation. I rebooted the server
but it caused kernel panic. Rebooted again and it seems working now.

Postfix started logging this repeatedly after installation of
MegaMonitor:
> Jun 26 11:36:43 server1 postfix/master[162]: warning: process /usr/local/lib
> exec/postfix/proxymap pid 49073 killed by signal 11
> Jun 26 11:36:43 server1 postfix/master[162]: warning: /usr/local/libexec/pos
> tfix/proxymap: bad command startup -- throttling

kernel log:
> pid 49073 (proxymap), uid 0: exited on signal 11 (core dumped)
> pid 49075 (proxymap), uid 0: exited on signal 11 (core dumped)
> pid 49076 (proxymap), uid 0: exited on signal 11 (core dumped)
> .
> .

I tried restarting Postfix but it would not listen smtp any more. I
rebooted the server and got the following logs:

> Fatal trap 12: page fault while in kernel mode
> mp_lock = 00000002; cpuid = 0; lapic.id = 00000000
> fault virtual address	= 0x30
> fault code		= supervisor read, page not present
> instruction pointer	= 0x8:0xc022a660
> stack pointer	        = 0x10:0xd739cd8c
> frame pointer	        = 0x10:0xd739cd8c
> code segment		= base 0x0, limit 0xfffff, type 0x1b
> 			= DPL 0, pres 1, def32 1, gran 1
> processor eflags	= interrupt enabled, resume, IOPL = 0
> current process		= 250 (httpd)
> interrupt mask		= none <- SMP: XXX
> trap number		= 12
> panic: page fault
> mp_lock = 00000002; cpuid = 0; lapic.id = 00000000
> boot() called on cpu#0
> 
> syncing disks... 57 18 7 6 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 
> giving up on 2 buffers
> Uptime: 53s
> Automatic reboot in 15 seconds - press a key on the console to abort
> WARNING: / was not properly dismounted

Then I rebooted the server again. fsck completed successfully and
server1 returned to the normal state.


Server 2:

Started reporting filesystem inconsistency after installation of
MegaMonitor. egrep core dumps everytime it runs.

daily run output:
> find: /usr/src/contrib/libstdc++/std/bastring.cc: Bad file descriptor
> find: /usr/src/contrib/libstdc++/std/bastring.h: Bad file descriptor
> find: /usr/src/contrib/libstdc++/std/complext.cc: Bad file descriptor
> .
> .
> find: /usr/src/contrib/libstdc++/stl/function.h: Bad file descriptor

fsck:
> ** /dev/amrd0s1f (NO WRITE)
> ** Last Mounted on /usr
> ** Phase 1 - Check Blocks and Sizes
> PARTIALLY ALLOCATED INODE I=518852
> UNEXPECTED SOFT UPDATE INCONSISTENCY
> 
> PARTIALLY ALLOCATED INODE I=518861
> UNEXPECTED SOFT UPDATE INCONSISTENCY
> 
> PARTIALLY ALLOCATED INODE I=518875
> UNEXPECTED SOFT UPDATE INCONSISTENCY
> 
> ** Phase 2 - Check Pathnames
> UNALLOCATED  I=518849  OWNER=root MODE=0
> SIZE=0 MTIME=Jan  1 09:00 1970
> NAME=/src/contrib/libstdc++/std/bastring.cc
> 
> UNEXPECTED SOFT UPDATE INCONSISTENCY
> 
> UNALLOCATED  I=518850  OWNER=root MODE=0
> SIZE=0 MTIME=Jan  1 09:00 1970
> NAME=/src/contrib/libstdc++/std/bastring.h
> 
> UNEXPECTED SOFT UPDATE INCONSISTENCY
> 
> UNALLOCATED  I=518851  OWNER=root MODE=0
> SIZE=0 MTIME=Jan  1 09:00 1970
> NAME=/src/contrib/libstdc++/std/complext.cc
> 
> UNEXPECTED SOFT UPDATE INCONSISTENCY
> .
> .
> UNALLOCATED  I=518879  OWNER=root MODE=0
> SIZE=0 MTIME=Jan  1 09:00 1970
> NAME=/src/contrib/libstdc++/stl/function.h
> 
> UNEXPECTED SOFT UPDATE INCONSISTENCY
> 
> ** Phase 3 - Check Connectivity
> ** Phase 4 - Check Reference Counts
> LINK COUNT FILE I=2060810  OWNER=root MODE=0
> SIZE=0 MTIME=Jul  2 10:44 2003  COUNT 0 SHOULD BE -1
> 
> ** Phase 5 - Check Cyl groups
> FREE BLK COUNT(S) WRONG IN SUPERBLK
> 
> SUMMARY INFORMATION BAD
> 
> BLK(S) MISSING IN BIT MAPS
> 
> 166647 files, 739567 used, 14837246 free (49006 frags, 1848530 blocks, 0.3% frag
> mentation)

kernel log:
> Jun 30 03:01:45 server2 /kernel: pid 1271 (egrep), uid 0: exited on signal 11 (core dumped)
> Jun 30 03:01:45 server2 /kernel: pid 1289 (egrep), uid 0: exited on signal 11 (core dumped)
> Jun 30 04:50:00 server2 /kernel: pid 1475 (egrep), uid 0: exited on signal 11 (core dumped)
> .
> .

I'm unable to fix nor remove those corrupted files. I also tried fsck
and clri with any possible options but could not free the inodes.

MegaMonitor itself is working fine.

-- 
Shuji Kono <shuji.kono at oriental.ac>