[mfi] COMMAND 0x.. TIMEOUT AFTER ## SECONDS

Thu May 30 02:37:32 UTC 2013

Hello all,

I've had 9.1-R running on a few Dell M620 blades (with H710 controllers) in them for a bit now and have had command timeout errors showing up from time to time.  The system appears to still be responsive, although, I have noticed a couple of times where disk I/O will seem to pause for a few seconds.  No panics, nothing forcing me to restart.

sbruno@ reported that he was running R620s with H710P cards in them (not blades) with A02 (21.1.0-0007) firmware and was not running into this issue.  I downgraded one of my systems to the same firmware, but still ran into timeouts.  Note H710 versus H710P.

If anyone has any recommendations or next steps, please let me know.  I'd be happy to open a PR if desired.  I've included details below.

Thanks,
-r

-- 
Ryan Steinmetz
PGP: EF36 D45A 5CA9 28B1 A550  18CD A43C D111 7AD7 FAF2

# mfiutil show adapter
mfi0 Adapter:
     Product Name: PERC H710 Mini
    Serial Number: 31A00ZD
         Firmware: 21.2.0-0007
      RAID Levels: JBOD, RAID0, RAID1, RAID5, RAID6, RAID10, RAID50
   Battery Backup: present
            NVRAM: 32K
   Onboard Memory: 512M
   Minimum Stripe: 64k
   Maximum Stripe: 1M
# uname -rm
9.1-RELEASE-p3 amd64
# pciconf -lv
mfi0 at pci0:2:0:0:        class=0x010400 card=0x1f371028 chip=0x005b1000 rev=0x05 hdr=0x00
     vendor     = 'LSI Logic / Symbios Logic'
     device     = 'MegaRAID SAS 2208 [Thunderbolt]'
     class      = mass storage
     subclass   = RAID
# dmesg | grep mfi0
mfi0: 1428 (422722487s/0x0020/info) - Shutdown command received from host
mfi0: 1429 (boot + 4s/0x0020/info) - Firmware initialization started (PCI ID 005b/1000/1f37/1028)
mfi0: 1430 (boot + 4s/0x0020/info) - Firmware version 3.130.05-2086
mfi0: 1431 (boot + 5s/0x0008/info) - Battery Present
mfi0: 1432 (boot + 5s/0x0020/info) - Package version 21.2.0-0007
mfi0: 1433 (boot + 5s/0x0020/info) - Board Revision A00
mfi0: 1434 (boot + 6s/0x0008/info) - Battery temperature is normal
mfi0: 1435 (boot + 6s/0x0008/info) - Current capacity of the battery is above threshold
mfi0: 1436 (boot + 20s/0x0004/info) - Enclosure PD 20(c None/p1) communication restored
mfi0: 1437 (boot + 20s/0x0002/info) - Inserted: Encl PD 20
mfi0: 1438 (boot + 20s/0x0002/info) - Inserted: PD 20(c None/p1) Info: enclPd=20, scsiType=d, portMap=00, sasAddr=5948f090ebf23500,0000000000000000
mfi0: 1439 (boot + 20s/0x0002/info) - Inserted: PD 00(e0x20/s0)
mfi0: 1440 (boot + 20s/0x0002/info) - Inserted: PD 00(e0x20/s0) Info: enclPd=20, scsiType=0, portMap=00, sasAddr=50000c0f02c1bab6,0000000000000000
mfi0: 1441 (boot + 20s/0x0002/info) - Inserted: PD 01(e0x20/s1)
mfi0: 1442 (boot + 20s/0x0002/info) - Inserted: PD 01(e0x20/s1) Info: enclPd=20, scsiType=0, portMap=01, sasAddr=50000c0f026bb0d2,0000000000000000
mfi0: 1443 (422722572s/0x0020/info) - Time established as 05/24/13 14:56:12; (45 seconds since power on)
mfi0: 1444 (422722598s/0x0008/info) - Battery started charging
mfi0: 1445 (422722793s/0x0008/info) - Battery charge complete
mfi0: 1446 (422722801s/0x0020/info) - Host driver is loaded and operational
mfid0 on mfi0
mfid0: 857856MB (1756889088 sectors) RAID volume (no label) is optimal
Trying to mount root from ufs:/dev/mfid0p3 [rw]...
mfi0: 1447 (422723530s/0x0002/WARN) - PD 00(e0x20/s0) Path 50000c0f02c1bab6  reset (Type 03)
mfi0: 1448 (422723530s/0x0002/WARN) - PD 01(e0x20/s1) Path 50000c0f026bb0d2  reset (Type 03)
mfi0: 1449 (422723530s/0x0002/info) - Unexpected sense: PD 01(e0x20/s1) Path 50000c0f026bb0d2, CDB: 2a 00 23 44 ea 00 00 00 80 00, Sense: 6/29/02
mfi0: 1450 (422723530s/0x0002/info) - Unexpected sense: PD 00(e0x20/s0) Path 50000c0f02c1bab6, CDB: 2a 00 23 44 ea 00 00 00 80 00, Sense: 6/29/02
mfi0: COMMAND 0xffffff8002b915b8 TIMEOUT AFTER 56 SECONDS
mfi0: COMMAND 0xffffff8002b906d8 TIMEOUT AFTER 56 SECONDS
mfi0: COMMAND 0xffffff8002b916c8 TIMEOUT AFTER 56 SECONDS
mfi0: COMMAND 0xffffff8002b92740 TIMEOUT AFTER 56 SECONDS
mfi0: COMMAND 0xffffff8002b92388 TIMEOUT AFTER 56 SECONDS
mfi0: COMMAND 0xffffff8002b928d8 TIMEOUT AFTER 46 SECONDS
mfi0: COMMAND 0xffffff8002b91750 TIMEOUT AFTER 45 SECONDS
mfi0: COMMAND 0xffffff8002b8fd48 TIMEOUT AFTER 45 SECONDS
mfi0: COMMAND 0xffffff8002b91f48 TIMEOUT AFTER 58 SECONDS
mfi0: COMMAND 0xffffff8002b92c08 TIMEOUT AFTER 58 SECONDS
mfi0: COMMAND 0xffffff8002b915b8 TIMEOUT AFTER 36 SECONDS
mfi0: 899 (422741297s/0x0002/WARN) - Encl PD 20 Path 5948f090ec1e9c00  reset (Type 03)
mfi0: 900 (422766000s/0x0020/info) - Patrol Read started
mfi0: 901 (422775083s/0x0020/info) - Patrol Read complete
mfi0: 902 (422790275s/0x0002/WARN) - Encl PD 20 Path 5948f090ec1e9c00  reset (Type 03)
mfi0: COMMAND 0xffffff8002b90870 TIMEOUT AFTER 42 SECONDS
mfi0: COMMAND 0xffffff8002b8f5d8 TIMEOUT AFTER 34 SECONDS
mfi0: 903 (422852812s/0x0002/WARN) - PD 01(e0x20/s1) Path 50000c0f020c5e26  reset (Type 03)
mfi0: 904 (422852812s/0x0002/WARN) - Encl PD 20 Path 5948f090ec1e9c00  reset (Type 03)
mfi0: 905 (422852818s/0x0002/info) - Unexpected sense: PD 01(e0x20/s1) Path 50000c0f020c5e26, CDB: 2a 00 00 00 01 22 00 00 08 00, Sense: 6/29/02
mfi0: COMMAND 0xffffff8002b905c8 TIMEOUT AFTER 31 SECONDS
mfi0: COMMAND 0xffffff8002b8f330 TIMEOUT AFTER 31 SECONDS
mfi0: COMMAND 0xffffff8002b929e8 TIMEOUT AFTER 43 SECONDS
mfi0: COMMAND 0xffffff8002b91310 TIMEOUT AFTER 43 SECONDS
mfi0: COMMAND 0xffffff8002b90540 TIMEOUT AFTER 38 SECONDS
mfi0: COMMAND 0xffffff8002b8f660 TIMEOUT AFTER 38 SECONDS
mfi0: COMMAND 0xffffff8002b92fc0 TIMEOUT AFTER 38 SECONDS
mfi0: COMMAND 0xffffff8002b8f110 TIMEOUT AFTER 38 SECONDS
mfi0: COMMAND 0xffffff8002b910f0 TIMEOUT AFTER 38 SECONDS
mfi0: COMMAND 0xffffff8002b91a80 TIMEOUT AFTER 38 SECONDS
mfi0: COMMAND 0xffffff8002b90e48 TIMEOUT AFTER 34 SECONDS
mfi0: COMMAND 0xffffff8002b90a90 TIMEOUT AFTER 34 SECONDS
mfi0: COMMAND 0xffffff8002b91640 TIMEOUT AFTER 34 SECONDS
mfi0: COMMAND 0xffffff8002b92960 TIMEOUT AFTER 40 SECONDS
mfi0: COMMAND 0xffffff8002b92960 TIMEOUT AFTER 70 SECONDS
mfi0: COMMAND 0xffffff8002b92740 TIMEOUT AFTER 32 SECONDS
mfi0: COMMAND 0xffffff8002b928d8 TIMEOUT AFTER 32 SECONDS
mfi0: COMMAND 0xffffff8002b8fa18 TIMEOUT AFTER 31 SECONDS
mfi0: COMMAND 0xffffff8002b93268 TIMEOUT AFTER 51 SECONDS
mfi0: COMMAND 0xffffff8002b8f5d8 TIMEOUT AFTER 51 SECONDS
mfi0: COMMAND 0xffffff8002b8f198 TIMEOUT AFTER 51 SECONDS
mfi0: COMMAND 0xffffff8002b92630 TIMEOUT AFTER 51 SECONDS
mfi0: COMMAND 0xffffff8002b8f3b8 TIMEOUT AFTER 51 SECONDS
mfi0: COMMAND 0xffffff8002b8f000 TIMEOUT AFTER 51 SECONDS
mfi0: COMMAND 0xffffff8002b92740 TIMEOUT AFTER 51 SECONDS
mfi0: COMMAND 0xffffff8002b928d8 TIMEOUT AFTER 51 SECONDS
mfi0: COMMAND 0xffffff8002b8fa18 TIMEOUT AFTER 42 SECONDS
mfi0: COMMAND 0xffffff8002b8fd48 TIMEOUT AFTER 42 SECONDS
mfi0: 906 (422962690s/0x0002/WARN) - Encl PD 20 Path 5948f090ec1e9c00  reset (Type 03)
mfi0: 907 (423017849s/0x0002/WARN) - Encl PD 20 Path 5948f090ec1e9c00  reset (Type 03)
mfi0: 908 (423040626s/0x0002/WARN) - Encl PD 20 Path 5948f090ec1e9c00  reset (Type 03)
mfi0: COMMAND 0xffffff8002b904b8 TIMEOUT AFTER 31 SECONDS