problem with 9k jumbo clusters

[Alexander V. Chernikov]

On 04.12.2014 13:50, Yuriy Tabolin wrote:
> Hi All.
> I have a server with two Intel 10G NIC. OS FreeBSD 10.1-Release amd64. 
> Server works like NFS, samba-server and iSCSI target. Both NICs 
> aggregated into lagg device and set MTU 9014 to them. There are some 
> tuning sysctl.conf:
> kern.maxfiles=6289601
> kern.maxfilesperproc=5660640
> kern.maxvnodes=3339565
> kern.ipc.nmbclusters=12255588
> kern.ipc.nmbjumbop=6127794
> kern.ipc.nmbufs=78435780
> kern.ipc.maxsockbuf=16777216
> kern.ipc.maxsockets=6289600
> net.inet.tcp.sendbuf_max=16777216
> net.inet.tcp.sendspace=65536
> net.inet.tcp.recvbuf_max=16777216
> net.inet.tcp.recvspace=65536
>
> After some days of working, the errors are appearing:
> ix1: Interface stopped DISTRIBUTING, possible flapping
> ix0: Interface stopped DISTRIBUTING, possible flapping
> ix0: Could not setup receive structures
> ix1: Could not setup receive structures
Hello. It looks like
https://lists.freebsd.org/pipermail/freebsd-net/2014-May/038630.html
is relevant here.
>
> After that errors the NICs stoped working. netstat -m shows:
> 32881/33854/66735 mbufs in use (current/cache/total)
> 16370/8198/24568/12255588 mbuf clusters in use (current/cache/total/max)
> 16370/4807 mbuf+clusters out of packet secondary zone in use 
> (current/cache)
> 0/873/873/6127794 4k (page size) jumbo clusters in use 
> (current/cache/total/max)
> 16383/21517/37900/1815641 9k jumbo clusters in use 
> (current/cache/total/max)
> 0/0/0/1021298 16k jumbo clusters in use (current/cache/total/max)
> 188407K/222004K/410411K bytes allocated to network (current/cache/total)
> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
> 0/101414306/0 requests for jumbo clusters denied (4k/9k/16k)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
>
> 9k jumbo clusters max is too big, but looks like system cannot 
> allocate them. There are huge number of "9k requests for jumbo 
> clusters denied". ifconfig ix down/up don't helped, reboot is needed. 
> Thanks for any help!
>
>
> -- 
> Best regards,
> Tabolin Yuriy
> System administrator
> Speech Technology Center
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>