svn commit: r273112 - head/sys/dev/ixgbe

[Adrian Chadd]

Hm, are there any other hardware counters that we can use to see what
was dropped?

I didn't see performance drops doing this; in fact I saw performance
increases with FC disabled and DROP_EN set.

Thanks,

-adrian


On 6 November 2014 03:05, Alexander V. Chernikov <melifaro at freebsd.org> wrote:
> On 15.10.2014 05:22, Adrian Chadd wrote:
>>
>> Author: adrian
>> Date: Wed Oct 15 01:22:56 2014
>> New Revision: 273112
>> URL: https://svnweb.freebsd.org/changeset/base/273112
>>
>> Log:
>>    Set the DROP_EN bit before the RX queue is brought up and active.
>>       He noticed issues setting this bit in SRRCTL after the queue was up,
>>    so doing it from the sysctl handler isn't enough and may not actually
>>    work correctly.
>
> While investigating/measuring ixgbe performance w/ and without flow control
> I noticed that disabling fc entirely (e.g. disabling 512K skid buffer and
> turning drop_en on)
> not only makes things a bit worse, but also makes NIC stop accounting
> tail-drops as errors
> at all.
> It is a bit unhandy with HW counters, since you, for example, see 2M packets
> being received
> in netstat, but in reality system receives only 1M (And there is no easy way
> to determine exact
> count).
>
>>       This commit doesn't remove the sysctl path or try to change its
>>    behaviour.  I'll talk with others about how to finish fixing that
>>    before I tackle that.
>>       PR:               kern/194311
>>    Submitted by:        luigi
>>    MFC after:   3 days
>>    Sponsored by:        Norse Corp, Inc
>>
>> Modified:
>>    head/sys/dev/ixgbe/ixgbe.c
>>
>> Modified: head/sys/dev/ixgbe/ixgbe.c
>>
>> ==============================================================================
>> --- head/sys/dev/ixgbe/ixgbe.c  Wed Oct 15 01:16:11 2014        (r273111)
>> +++ head/sys/dev/ixgbe/ixgbe.c  Wed Oct 15 01:22:56 2014        (r273112)
>> @@ -4377,6 +4377,20 @@ ixgbe_initialize_receive_units(struct ad
>>                 srrctl &= ~IXGBE_SRRCTL_BSIZEPKT_MASK;
>>                 srrctl |= bufsz;
>>                 srrctl |= IXGBE_SRRCTL_DESCTYPE_ADV_ONEBUF;
>> +
>> +               /*
>> +                * Set DROP_EN iff we have no flow control and >1 queue.
>> +                * Note that srrctl was cleared shortly before during
>> reset,
>> +                * so we do not need to clear the bit, but do it just in
>> case
>> +                * this code is moved elsewhere.
>> +                */
>> +               if (adapter->num_queues > 1 &&
>> +                   adapter->hw.fc.requested_mode == ixgbe_fc_none) {
>
> This code seems to turn drop_en always on HEAD due to
> hw->fc.requested_mode is inherited from adapter->fc at the end of
> ixgbe_init_locked(), _after_ calling ixgbe_initialize_receive_units()
>
>> +                       srrctl |= IXGBE_SRRCTL_DROP_EN;
>> +               } else {
>> +                       srrctl &= ~IXGBE_SRRCTL_DROP_EN;
>> +               }
>> +
>>                 IXGBE_WRITE_REG(hw, IXGBE_SRRCTL(i), srrctl);
>>                 /* Setup the HW Rx Head and Tail Descriptor Pointers */
>>
>>
>