Changes in the network interface queueing handoff model

[Robert Watson]

On Mon, 31 Jul 2006, John Polstra wrote:

>> Attached is a patch that maintains the current if_start, but adds 
>> if_startmbuf.  If a device driver implements if_startmbuf and the global 
>> sysctl net.startmbuf_enabled is set to 1, then the if_startmbuf path in the 
>> driver will be used.  Otherwise, if_start is used.  I have modified the 
>> if_em driver to implement if_startmbuf also.  If there is no packet backlog 
>> in the if_snd queue, it directly places the packet in the transmit 
>> descriptor ring. If there is a backlog, it uses the if_snd queue protected 
>> by driver mutex, rather than a separate ifq mutex.
>
> I question whether you need a fallback software if_snd queue at all for 
> modern devices such as the Intel and Broadcom gigabit chips.  The hardware 
> transmit descriptor rings typically have sizes of the order of 256 
> descriptors.  I think if the ring fills up, you could simply drop the packet 
> with ENOBUFS.  That's what happens if the if_snd queue fills up, and its 
> maximum size is comparable to the sizes of modern descriptor rings.  It 
> would simplify things quite a bit to eliminate the if_snd queue entirely for 
> such devices.

I tend to agree, but implemented full queueing support for if_em to make sure 
I understood to complexity implications of completely removing queueing from 
the ifnet side dispatch.  I guess an interesting question for us is how we 
decide what the right threshold is to implement software queuing.  Do any 
if_em cards need software queueing, or do they all have adequate in-hardware 
queues as is?  Entirely cutting the queue code would significantly simplify 
em_startmbuf.

Robert N M Watson
Computer Laboratory
University of Cambridge