ixl 40G bad performance?

[Luigi Rizzo]

On Thu, Dec 10, 2015 at 10:40 AM, Adrian Chadd <adrian.chadd at gmail.com> wrote:
> On 10 December 2015 at 10:29, Denis Pearson <dennix.pearson at gmail.com> wrote:
>> On Thu, Dec 10, 2015 at 2:18 PM, Eggert, Lars <lars at netapp.com> wrote:
>>
>>> On 2015-10-26, at 18:40, Eggert, Lars <lars at netapp.com> wrote:
>>> > On 2015-10-26, at 17:08, Pieper, Jeffrey E <jeffrey.e.pieper at intel.com>
>>> wrote:
>>> >> As a caveat, this was using default netperf message sizes.
>>> >
>>> > I get the same ~3 Gb/s with the default netperf sizes and driver 1.4.5.
>>>
>>> Now there is version 1.4.8 on the Intel website, but it doesn't change
>>> things for me.
>>>
>>
>> I had the opportunity to see similar numbers and behavior while using XL710
>> 1.4.3 as of FreeBSD r291085 while in DPDK poll mode, but driver 1.2.8 as of
>> r292035 was providing expected numbers. While removing rxcsum/txcsum did
>> not provide differences, fully removing RSS + disabling rx/cxsum support
>> provided better numbers.
>
> Can someone debug this a bit more? (My kit with ixl NICs in it is
> still not up and available. :( )
>
> Device RSS, even without kernel RSS enabled, shouldn't cause a massive
> performance drop. If it is then something else odd is going on.

I am not sure whether we are digressing (Lars' complaint was about
poor bulk throughput, now i see DPDK and high packet rates mentioned
so i feel obliged to pitch in!) but a related piece of info:

last spring, with netmap and i40e on linux (don't remember
which driver/firmware), we saw that enabling FlowDirector
killed the pps throughput (from 32 down to 18 Mpps).

FlowDirector is a device feature which was probably affecting
ordinary processing on the NIC, either because of bugs or because
of consuming controller resources.
The same may be possibly happening with other device features.

cheers
luigi

>
> Do you have a diff where you removed things?
>
>
> -adrian
>
>> However now with driver 1.4.8 and the same set of hardware setup, except
>> for a different transceiver, I can get 36Gbps/24Mpps with no further
>> tweaks, so if you can replace your transceiver, shall be a different test
>> as a starting point.
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"



-- 
-----------------------------------------+-------------------------------
 Prof. Luigi RIZZO, rizzo at iet.unipi.it  . Dip. di Ing. dell'Informazione
 http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
 TEL      +39-050-2217533               . via Diotisalvi 2
 Mobile   +39-338-6809875               . 56122 PISA (Italy)
-----------------------------------------+-------------------------------