bxe driver crash -- 'client ramrod halt failed!'

[camden lindsay]

Hello-
Please excuse if i am not following customary protocol, i've worked very
little with bsd and linux mailing lists.
My friend and I are trying to set up a Broadcom Netxtreme II NIC (57711 A0)
in a BSD box, and we are having problems.

Problem summary:
BSD machine appears to boot healthily, and interface configures as expected
using ifconfig
BSD and Windows machines can ping each other, as  expected
BSD machine goes unreachable after first tcp packets arrive on interface,
be it via SCP/SSH, HTTP/HTTPS or Iperf
Ifconfig down after this hang leads to error 'client ramrod halt failed!'.
Ifconfig up after ramrod failure bombards stdout with error messages
(attached) seeming to indicate receive registers expected to be 0/empty
have data in them.
Identical error messages repeat until the device is once again ifconfig'd
down

UDP iperf works prior to any TCP traffic, albeit seems slower than should
be.

To verify hardware, the same card was tested in the same hardware
environment using knoppix (7.0.4 DVD), and performed as expected using
iperf with TCP.

Environment:
-Windows 7 Machine
--Solar Flare 4000

-BSD Machine
--Nas4Free 9.1.0.1 rev 358 (BSD 9.1 RC2)
[blake at storage ~]$ uname -a
FreeBSD storage.local 9.1-RC2 FreeBSD 9.1-RC2 #0: Mon Oct  8 03:51:59 JST
2012     aoyama at nas4free.local:/usr/obj/nas4free/usr/src/sys/NAS4FREE-amd64
 amd64
and
--BSD 9.1. RC2 vanilla  (both display same symptoms)


--Broadcom NetExtreme 57711 A0
blake at storage ~]$ dmesg | grep bxe
bxe0: <Broadcom NetXtreme II BCM57711 10GbE (A0) BXE v:1.5.52
bxe0: Ethernet address: 00:10:18:b8:e4:5c
bxe0: ASIC (0x164F0000); Rev (A0); Bus (PCIe x8, 5Gbps); Flags (MSI-X);
Queues (RSS:4); BD's (RX:510,TX:255); Firmware (5.2.13); Bootcode (6.2.10)
bxe1: <Broadcom NetXtreme II BCM57711 10GbE (A0) BXE v:1.5.52
bxe1: Ethernet address: 00:10:18:b8:e4:5e
bxe1: ASIC (0x164F0000); Rev (A0); Bus (PCIe x8, 5Gbps); Flags (MSI-X);
Queues (RSS:4); BD's (RX:510,TX:255); Firmware (5.2.13); Bootcode (6.2.10)


[blake at storage ~]$ pciconf -lv  | grep bxe -A 4
bxe0 at pci0:1:0:0: class=0x020000 card=0x111314e4 chip=0x164f14e4 rev=0x00
hdr=0x00
    vendor     = 'Broadcom Corporation'
    device     = 'NetXtreme II BCM57711 10-Gigabit PCIe'
    class      = network
    subclass   = ethernet
bxe1 at pci0:1:0:1: class=0x020000 card=0x111314e4 chip=0x164f14e4 rev=0x00
hdr=0x00
    vendor     = 'Broadcom Corporation'
    device     = 'NetXtreme II BCM57711 10-Gigabit PCIe'
    class      = network
    subclass   = ethernet


Any pointers toward troubleshooting would be greatly appreciated.
Thanks!
Camden
-------------- next part --------------
kernel: bxe1: /usr/src/sys/dev/bxe/if_bxe.c(6854): FATAL hardware block attention (set1 = 0x08000000)!\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ---------- Begin crash dump ----------\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ------------------------------ Idle Check ------------------------------\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING PXP2: VQ1 is not empty. Value is 0x3\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING PXP2: VQ9 is not empty. Value is 0x2\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR CFC: AC > 1 - LCID 0 CID_CAM 0x80000 Value is 0x10\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING UCM: XX protection CAM is not empty. Value is 0x1\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING BRB1: BRB is not empty. Value is 0x5\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR BRB1: interrupt status is not 0. Value is 0x40400\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR UPB: interrupt status is not 0. Value is 0x2\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: INFO PXP2: interrupt status 1 is not 0. Value is 0x20\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR AEU: P0 AFTER_INVERT_2 is not 0. Value is 0x8000000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR AEU: P1 AFTER_INVERT_2 is not 0. Value is 0x8000000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR AEU: MCP AFTER_INVERT_2 is not 0. Value is 0x8000000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING UCM: XX_FREE is not 27. Value is 0x17\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING PRS: SERIAL_NUM_STATUS_LSB is not 0. Value is 0xfff80000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING PRS: SERIAL_NUM_STATUS_MSB is not 0. Value is 0x1edff7\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: INFO PXP2: parity status 0 is not 0. Value is 0x10\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR USEM: interrupt status 0 is not 0. Value is 0x10000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: bxe_idle_chk(): Failed with 7 error(s) and 2 warning(s)!\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ------------------------------------------------------------------------\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ------------------------------ Idle Check ------------------------------\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING PXP2: VQ1 is not empty. Value is 0x3\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING PXP2: VQ9 is not empty. Value is 0x2\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR CFC: AC > 1 - LCID 0 CID_CAM 0x80000 Value is 0x10\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING UCM: XX protection CAM is not empty. Value is 0x1\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING BRB1: BRB is not empty. Value is 0x5\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR BRB1: interrupt status is not 0. Value is 0x40400\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR UPB: interrupt status is not 0. Value is 0x2\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: INFO PXP2: interrupt status 1 is not 0. Value is 0x20\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR AEU: P0 AFTER_INVERT_2 is not 0. Value is 0x8000000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR AEU: P1 AFTER_INVERT_2 is not 0. Value is 0x8000000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR AEU: MCP AFTER_INVERT_2 is not 0. Value is 0x8000000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING UCM: XX_FREE is not 27. Value is 0x17\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING PRS: SERIAL_NUM_STATUS_LSB is not 0. Value is 0xfff80000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: WARNING PRS: SERIAL_NUM_STATUS_MSB is not 0. Value is 0x3edff7\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: INFO PXP2: parity status 0 is not 0. Value is 0x10\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ERROR USEM: interrupt status 0 is not 0. Value is 0x10000\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: bxe_idle_chk(): Failed with 7 error(s) and 2 warning(s)!\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ------------------------------------------------------------------------\par
Nov 17 21:57:57\tab storage\tab kernel: bxe1: ---------- End crash dump ----------\par
}