Debugging 3Ware 9000 series hangs under load

[adam radford]

On Mon, Jul 7, 2008 at 4:26 PM, Mike Andrews <mandrews at bit0.com> wrote:
> I've occasionally had problems with 3Ware 9550SX, 9650SE, and now 9690SA
> cards hanging under load.  By "hanging" I mean "swap_pager: indefinite wait
> buffer" messages on the console, machine still pingable, etc -- basically a
> livelock situation.  "Heavy load" usually involves a busy MySQL instance
> combined with, say, copying some multi-gigabyte files, maybe also combined
> with rsync, all running concurrently.  The problem is the hangs are
> extremely sporadic and not easily reproducible on demand. It seems to happen
> with both ZFS and UFS2; I've been sticking to UFS2 for this for now due to
> some weird ZFS+MySQL issues (which are annoying but not relevant to this
> particular problem)...
>
> On a FreeBSD/amd64 7-STABLE system built from less-than-one-week-old source,
> with serial console and the kernel debugger compiled in, but being a
> complete idiot on how to use KDB, can someone tell me what would be the most
> useful info I could gather from KDB the next time this happens?

Mike,

Can you first run the 3ware command line utility (tw_cli) and post the
results of:    tw_cli info c0 diag

-Adam