shutdown -r / shutdown -h / reboot all hang and don't cleanly dismount

[Adam Strohl]

Hello -STABLE@,

So I've seen this situation seemingly randomly on a number of both 
physical 9.1 boxes as well as VMs for I would say 6-9 months at least. 
  I finally have a physical box here that reproduces it consistently 
that I can reboot easily (ie; not a production/client server).

No matter what I do:

reboot
shutdown -p
shutdown -r

This specific server will stop at "All buffers synced" and not actually 
power down or reboot.  KB input seems to be ignored.  This server is a 
ZFS NAS (with GMIRROR for boot blocks) but the other boxes which show 
this are using GMIRRORs for root/swap/boot (no ZFS).

Here is what happens on the console: http://i.imgur.com/1H8JMyB.jpg

When I reset the server it appears that disks were not dismounted 
cleanly ... on this ZFS box it comes back quick because ZFS is good like 
that but on the other servers with GMIRROR roots rebuilding the GMIRROR 
and fscking at the same time is murder on the disk/performance until it 
finishes.

Another interesting thing is that this particular server runs slapd 
(OpenLDAP) which, when it comes back up, has a "corrupted" DB (easily 
fixed with db_recover, but still).  This might be because FS commits 
aren't happening at the end.   I can even manually stop slapd (service 
slapd stop) then run sync(8) (I assume this does something for ZFS too) 
and it still comes back as hosed if I reboot shortly after.  If I 
start/stop slapd it's fine.  So I feel like there is an FS/dismount 
thing going on here.

Additional information: I also have some boxes which will reboot (ie; 
they don't freeze like some do at the end) but they don't dismount 
cleanly either and have to rebuild both GMIRROR and fsck.  This might be 
a different issue, too.

Anyone have any thoughts?  Let me know if I can provide more details etc.

-- 
Adam Strohl
http://www.ateamsystems.com/