Issues with bxe NIC

[Marius Halden]

On Thu, Feb 1, 2018, at 15:17, Marius Halden wrote:
> Hi,
> 
> We have two routers running FreeBSD r320487 (BSDRP) with bxe NICs, they 
> are connected together with a twinax cable and to our ISP with 1Gbit 
> fiber. The NICs in question are Broadcom BCM57840S (Supermicro AOC-STG-
> B4S).
> 
> The link between the routers will go down on one side not long after 
> it's brought up while staying up on the other side. I am not able to 
> bring the link back up without either unplugging the cable or powering 
> the routers off and on again, if I try to bring them up by taking them 
> down and up with ifconfig they will stay down no matter what the state 
> was before.
> 
> After the routers have been up some time the console will be flooded 
> with messages like this:
> 
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ERROR: resource (0x0) in use (status 0xffffffff bit 0x1)
> bxe0: ELINK EVENT LOG (3)
> bxe0: ELINK EVENT LOG (3)
> 
> This is not always limited to just one interface, but can include all 
> interfaces which are in use.

Trying to bringing down the interface with ifconfig, the following was logged:

bxe0: ERROR: ECORE: timeout waiting for state 0
bxe0: ERROR: Failed to delete MACs (-4) mac_type 0 wait_for_comp 0x1
bxe0: ERROR: Failed to delete UC MACs list (-4)
bxe0: ERROR: FW failed to respond!
bxe0: ERROR: ECORE: timeout waiting for state 7
bxe0: ERROR: ECORE: timeout waiting for state 2
bxe0: ERROR: FUNC_STOP ramrod failed. Running a dry transaction (-4)
bxe0: ERROR: proper val not read from IGU!
bxe0: ERROR: ECORE: Unknown reset_phase (0x0) from MCP
bxe0: ERROR: FW failed to respond!
bxe0: ERROR: Parity error: HW block parity attention:
[0]:0x55540000 [1]:0x55555555 [2]:0x00005555 [3]:0xf0000000 [4]:0x00000028
bxe0: ERROR: resource (0xb) in use (status 0xffffffff bit 0x800)
bxe0: ERROR: resource (0xb) in use (status 0xffffffff bit 0x800)
bxe0: ERROR: resource (0xb) in use (status 0xffffffff bit 0x800)

Is this a hardware or a driver issue?

-- 
Marius Halden