Re: igc problems with heavy traffic

From: mike tancsa <mike_at_sentex.net>
Date: Fri, 05 Aug 2022 12:48:15 UTC
On 8/5/2022 2:19 AM, Jim King wrote:
> On 8/4/2022 12:29 PM, mike tancsa wrote:
>> On 8/4/2022 1:12 PM, mike tancsa wrote:
>>> On 7/31/2022 11:41 PM, Jim King wrote:
>>>> I'm trying to use a Qnap dual port 2.5Gb NIC with FreeBSD 13.1. 
>>>> FreeBSD recognizes the NIC and I'm able to connect to my LAN at 
>>>> 1Gb.  However, trying to push heavy traffic through the NIC results 
>>>> in a temporary loss of connectivity:  traffic stops and the message 
>>>> "igc0: link state changed to DOWN" appears in the system log. About 
>>>> 10 seconds later the message "igc0: link state changed to UP" is 
>>>> logged, followed by some messages from dhclient indicating that it 
>>>> has acquired a lease.  At that point network connectivity works 
>>>> normally, until I try to push a lot of traffic again.  I've been 
>>>> able to trigger this every time using iperf3 from another PC on my 
>>>> LAN, and by using Ookla's speedtest client (my Internet service is 
>>>> good for about 900 Mbps).
>>>>
>>> How long does it take to wedge the nic ? I have a pair of Topton 
>>> routers (4 IGC NICs) back to back and a simple iperf3 test seems to 
>>> work ok between the 2 units.
>>>
>> Actually, I was able to reproduce this too running iperf3 -P4 for 
>> about 60 seconds. Watching via sysutils/ifstat, it does indeed drop 
>> traffic when the link drops
>>
>>
> When I'm receiving traffic the wedge usually happens pretty quick. 
> iperf while connected at 1Gb (I get similar results at 2.5Gb):
>
If I fiddle with the Flow Control settings it seems to make a 
difference.  I can do it without bouncing the link.  I wonder what the 
Linux driver has as default ?

ie.

sysctl -w dev.igc.1.fc=1
sysctl -w dev.igc.0.fc=1

Does it help in your case ?

          * The possible values of the "fc" parameter are:
          *      0:  Flow control is completely disabled
          *      1:  Rx flow control is enabled (we can receive pause
          *          frames but not send pause frames).
          *      2:  Tx flow control is enabled (we can send pause frames
          *          frames but we do not receive pause frames).
          *      3:  Both Rx and Tx flow control (symmetric) is enabled.

          *  other:  No other values should be possible at this point.


I am also able to do

netsend 10.1.255.209 500 1400 190000 50

Sending packet of payload size 1400 every 0.000005263s for 50 seconds
calling time every 38 cycles

start:             1659703401.000000000
finish:            1659703451.000103336
send calls:        9500304
send errors:       0
approx send rate:  190006 pps
time/packet:       5263 ns
approx error rate: 0
waited:            820237403
approx waits/sec:  16404748
approx wait rate:  86

     ---Mike