A problem on TCP in High RTT Environment.

[Michael Tuexen]

On 11 Aug 2014, at 09:31, Niu Zhixiong <kaiaixi at gmail.com> wrote:

> During TCP benchmark.
> 
> root at Freetest0: # netstat -m
> 35806/2129/37935 mbufs in use (current/cache/total)
> 35251/1037/36288/1011638 mbuf clusters in use (current/cache/total/max)
> 35251/1030 mbuf+clusters out of packet secondary zone in use (current/cache)
> 0/6/6/505818 4k (page size) jumbo clusters in use (current/cache/total/max)
> 0/0/0/149872 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/84303 16k jumbo clusters in use (current/cache/total/max)
> 79481K/2630K/82111K bytes allocated to network (current/cache/total)
> 15/18307/31 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
> 
> During SCTP benchmark
> 36131/1804/37935 mbufs in use (current/cache/total)
> 35394/894/36288/1011638 mbuf clusters in use (current/cache/total/max)
> 35394/887 mbuf+clusters out of packet secondary zone in use (current/cache)
> 0/6/6/505818 4k (page size) jumbo clusters in use (current/cache/total/max)
> 0/0/0/149872 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/84303 16k jumbo clusters in use (current/cache/total/max)
> 79950K/2263K/82213K bytes allocated to network (current/cache/total)
> 15/18307/31 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
Thanks. I was more asking Lars to have a look if the igb problems he observed
are related to mbuf shortages... I observed that a long time ago and I think
it was fixed. Not sure about 10.0, since this was on head at that time...

Best regards
Michael
> 
> 
> Regards,
> Niu Zhixiong
> －－－－－－－－－－－－－－－
>  kaiaixi at gmail.com
> 
> 
> On Mon, Aug 11, 2014 at 3:17 PM, Michael Tuexen <Michael.Tuexen at lurchi.franken.de> wrote:
> On 11 Aug 2014, at 09:08, Eggert, Lars <lars at netapp.com> wrote:
> 
> > Hi,
> >
> > On 2014-8-10, at 5:48, Niu Zhixiong <kaiaixi at gmail.com> wrote:
> >> I am using Intel I350-T4 NIC.
> >
> > igb driver?
> >
> > I've been having weird issues with this driver under 10-RELEASE, too. On one machine, I had to limit hw.igb.num_queues=2 in order to get any sort of useful connectivity. On another machine, I had to severely bump kern.ipc.nmbclusters & friends. I'm not sure this is the issue here, since SCTP seems to be working OK, but I'm not trusting igb NICs at the moment.
> Was there any suspicious output provided by netstat -m when the problems occur?
> 
> Best regards
> Michael
> >
> > Lars
> 
>