Hang on boot in isp with QLA2342 after upgrading to 6.3

[Graham Allan]

On Thu, May 08, 2008 at 08:10:28PM -0500, Graham Allan wrote:
> Hi,
> 
> I've been trying to figure out a problem on a system which I just
> upgraded from FreeBSD 6.1 to 6.3. It's a Dell 1750 with a QLA2342
> (isp2312) HBA, connected to an EMC DS-16B2 (aka Brocade 3800) switch,
> and from there to a couple of AC&NC Jetstor arrays. It's all been
> working fine like this for some time under 6.1.
> 
> When I boot the system under 6.3, though I get a hang like this:
> 
> isp0: Interrupting Mailbox Command (0x6f) Timeout (500000us)
> isp0: PLOGI 0x031a00 failed
> isp0: Interrupting Mailbox Command (0x6e) Timeout (10000000us)
> isp0: Mailbox Command 'SEND SNS' failed (TIMEOUT)
> isp0: isp_pdb_sync: isp_scan_fabric failed
> isp0: Interrupting Mailbox Command (0x6e) Timeout (10000000us)
> isp0: Mailbox Command 'SEND SNS' failed (TIMEOUT)
> isp0: isp_pdb_sync: isp_scan_fabric failed
> isp0: Mailbox Command 'SEND SNS' failed (COMMAND ERROR)
> isp0: isp_pdb_sync: isp_scan_fabric failed
> 
> after which the system stays hung.

I've done some more testing on this and am left with a headache because
nothing makes sense! Would certainly be grateful if any fiber channel
gurus might comment.

I tested again with FreeBSD 7.0 and got the same result as above.

Next I tried swapping components in and out of the SAN to see if
different combinations have different results. These tests were with
7.0 since that was what I still had installed. The SAN switch has the
following devices connected:

1) AC&NC Jetstor 416F
2) AC&NC Jetstor 516F
3) Alphaserver ES40, Tru64 5.1B-6, KGPSA-CA (Emulex) HBA
4) Dell 1750, (the system I'm writing about), Qlogic 2342 HBA
5) Dell 1750, Windows 2003, LSI 7202P HBA

There's no zoning on the switch since all three servers connect to the
two Jetstors. Access to the RAID volumes is controlled by host filters
on the Jetstors.

So I tested with different combinations of devices connected to the
switch, with perplexing results:

FreeBSD + 416F only - boots fine
FreeBSD + 516F only - boots fine
FreeBSD + 416F + 516F - boots fine
FreeBSD + 416F + 516F + windows - boots fine
FreeBSD + 416F + 516F + ES40 - hangs with the above error
but, continuing...
FreeBSD + 516F + ES40 - boots fine
FreeBSD + 416F + ES40 - boots fine
FreeBSD + 416F + 516F + ES40 - hangs again

I can't make any sense of this... there are so many different systems
involved that there's no way to know where the problem really lies.
Although as it did work with FreeBSD 6.1, it feels to me like something
is wrong in the newer isp driver, but I have no solid knowledge to base
that on.

Thanks for any ideas,

Graham
-- 
-------------------------------------------------------------------------
Graham Allan - I.T. Manager - allan at physics.umn.edu - (612) 624-5040
School of Physics and Astronomy - University of Minnesota
-------------------------------------------------------------------------