FreeBSD 10.1-REL - network unaccessible after high traffic

[Mark Schouten]

Oh, didn't see your lowest remark. Then, the next thing that comes past here a few times per week is 'Try disabling TSO'.


Met vriendelijke groeten,

-- 
Kerio Operator in de Cloud? https://www.kerioindecloud.nl/
Mark Schouten  | Tuxis Internet Engineering
KvK: 61527076 | http://www.tuxis.nl/
T: 0318 200208 | info at tuxis.nl



 Van:   Cs <bimmer at field.hu> 
 Aan:   Mark Schouten <mark at tuxis.nl> 
 Cc:    <freebsd-net at freebsd.org> 
 Verzonden:   25-5-2015 11:12 
 Onderwerp:   Re: FreeBSD 10.1-REL - network unaccessible after high traffic 

It was on 1500 for ~3 years :) 
 
Regards, 
Csaba 
 
 
 
On May 25, 2015, 10:30, at 10:30, Mark Schouten <mark at tuxis.nl> wrote: 
> 
>Try lowering your mtu to 1500, that worked miracles for me.. 
> 
>--  
>Mark Schouten 
>Tuxis Internet Engineering 
>mark at tuxis.nl / 0318 200208 
> 
>> On 25 May 2015, at 09:36, "Cs" <bimmer at field.hu> wrote: 
>>  
>> Hi all, 
>>  
>> I have two FreeBSd 10.1-RELEASE servers connected to each other. They 
>were connected via cross link, but they are connected to a cisco switch 
>now (the problem was the same with cross link too). When transferring 
>huge files (50-500GB backup files) via Gigabit (it is important!) the 
>network randomly dies. The backup runs every day/week and sometimes the 
>connection is ok for months sometimes it happens twice a week. When the 
>network dies I can log in to the server via IPMI and use the console 
>everything is OK, but can't send anything out on the network. ifconfig 
>em0 down/up doesn't help nor netif restart. The problem never occured 
>when I used 100Mbit connection between them, but it was 3com NIC (xl), 
>gigabit adapter is Intel (em0). When I limit the transfer rate (rsync 
>bandwith limit or ipfw pipe) the problem is much more rare. 
>>  
>> I tried to set these tuning parameters on both servers with different 
>buffer size but nothing helped: 
>>  
>> # cat /etc/sysctl.conf 
>> security.bsd.see_other_uids=0 
>> net.inet.tcp.recvspace=512000 
>> net.route.netisr_maxqlen=2048 
>> kern.ipc.nmbclusters=1310720 
>> net.inet.tcp.sendbuf_max=16777216 
>> net.inet.tcp.recvbuf_max=16777216 
>> kern.ipc.soacceptqueue=32768 
>>  
>> # cat /boot/loader.conf 
>> geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) 
>> ipfw_load="YES" 
>> net.inet.ip.fw.default_to_accept=1 
>> kern.maxusers=4096 
>> accf_data_load="YES" 
>>  
>> The duplex settings are identical on both servers. 
>>  
>> Server A: 
>> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 
>9000 
>> 
>options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> 
> 
>>        ether 00:25:90:24:52:66 
>>        inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x 
>>        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> 
>>        media: Ethernet autoselect (1000baseT <full-duplex>) 
>>        status: active 
>>  
>> Server B: 
>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 
>9000 
>> 
>options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> 
> 
>>        ether 00:30:48:dd:fe:3e 
>>        inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x 
>>        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> 
>>        media: Ethernet autoselect (1000baseT <full-duplex>) 
>>        status: active 
>>  
>> Today I tried to set mtu to 9000 but in tcpdump I see that during scp 
>it is still 1500: 
>>    x.x.x.x.222 > x.x.x.x.37612: Flags [.], cksum 0xb6ee (incorrect -> 
>0xda6f), seq 35749, ack 113701596, win 7986, options [nop,nop,TS val 
>3103966325 ecr 853712893], length 0 
>> 09:27:33.912354 IP (tos 0x8, ttl 64, id 1028, offset 0, flags [DF], 
>proto TCP (6), length 1500) 
>> 09:27:33.912358 IP (tos 0x8, ttl 64, id 1029, offset 0, flags [DF], 
>proto TCP (6), length 1500) 
>>  
>>  
>> Any ideas? Thanks guys! 
>> _______________________________________________ 
>> freebsd-net at freebsd.org mailing list 
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net 
>> To unsubscribe, send any mail to 
>"freebsd-net-unsubscribe at freebsd.org" 
_______________________________________________ 
freebsd-net at freebsd.org mailing list 
http://lists.freebsd.org/mailman/listinfo/freebsd-net 
To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org" 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4549 bytes
Desc: Electronic Signature S/MIME
URL: <http://lists.freebsd.org/pipermail/freebsd-net/attachments/20150526/4ce9fb7a/attachment.bin>