individual queue blocking entire rx unit on ixgbe (Re: How do I balance bandwidth over several virtual NICs?)

[Luigi Rizzo]

On Wed, Oct 01, 2014 at 10:37:35AM -0700, Adrian Chadd wrote:
> Hi,
> 
> Try this on -HEAD. There were some recent fixes to ixgbe that haven't
> been RFCed.

I don't have a way to test this on HEAD.

Do you know if there is any change that could be related ?

On 10.1 beta 3 I noticed is that when I open a single queue on an ixgbe
(and with incoming traffic), the rx unit stalls no matter what the
previous state of dev.ix.*.fc (i.e. the DROP_EN bits) or QDE are.

In this state, toggling DROP_EN has no effect, whereas something that seems
effective is setting the QDE bit(s) in the PFQDE register for all queues,
_after_ i open the device.

>From the above my take is the following:

- on NIC reset, the SRRCTL register starts at 0 including DROP_EN;
  same goes for PFQDE.QDE

- setting SRRCTL.DROP_EN must happen before the receive unit is started;

- conversely, toggling PFQDE.QDE has effect even when the receive
  unit has started

- sysctl dev.ix.*.fc sets/clear DROP_EN but at the wrong time
  (the right time seems to be the window between reset and start)

I am going to run more tests to figure out.

cheers
luigi