cxl nic not working after reboot

[Navdeep Parhar]

On Mon, Aug 27, 2018 at 04:19:29PM +0200, Marius Halden wrote:
> Hi,
> 
> We have some routers with Chelsio T540-CR NICs using 1Gbps SFPs (1000
> Base-LX IIRC) to connect to our ISPs. After upgrading them to FreeBSD
> 11.2-p2 (BSDRP v1.91) we have run into some issues. When the machines
> are rebooted networking does not function at all through the ports
> connected with SFPs, ports connected with DAC cables work properly.
> According to ifconfig the SFPs are detected properly and they both

What exact ifconfig command do you use to bring the interfaces up?

Provide the output of these commands when the link isn't up:
# ifconfig -mvvv <interface>
# sysctl -n dev.t5nex.0.misc.devlog

> send and receive light as they should. If we remove the SFPs from the
> port and then reinsert them in the same port everything will start
> working as it should.
> 
> Has anyone else observed this problem? Any suggestions for what can be
> done to solve/work around this issue without actually going to the
> datacenter would be great.

Try ifconfig down/up instead of physically removing and then reinserting
the SFP module.  Does that work?

Regards,
Navdeep