svn commit: r277122 - projects/ifnet/sys/dev/msk

[John Baldwin]

On Friday, March 06, 2015 05:41:01 AM Gleb Smirnoff wrote:
> On Wed, Jan 14, 2015 at 05:43:58PM +0300, Gleb Smirnoff wrote:
> T> On Wed, Jan 14, 2015 at 09:33:04AM -0500, John Baldwin wrote:
> T> J> I posted some ideas about how to handle this in a thread several years
> T> J> ago on net@ with various alternatives.  In that case I was focused on
> T> J> buf_ring and I settled on an approach where a draining thread marked the
> T> J> queue as "busy" while it was draining it and cleared that flag before
> T> J> checking the head of the queue.  The enqueue process returned a
> T> J> different errno value (EINPROGRESS or some such) if it queued a packet
> T> J> into a "busy" queue and the transmit routines were changed to 1) always
> T> J> enqueue the packet, and 2) if EINPROGRESS wasn't returned, use a
> T> J> blocking mtx_lock and start transmitting.
> T> J> 
> T> J> However, even this model has some downsides in that one thread might be
> T> J> stuck transmitting packets queued by other threads and never pop back
> T> J> out to userland to service its associated application (a kind of
> T> J> starvation of the user side of the thread).  Note that the mtx_trylock
> T> J> approach has the same problem.  It might be nice to have a sort of limit
> T> J> on the number of packets a thread is willing to enqueue, but then you
> T> J> have the problem of ensuring any packets still on the queue when it hits
> T> J> its limit aren't also delayed indefinitely.
> T> 
> T> Thanks, I will try to code that.
> 
> John, can you please look at this patch? It is against projects/ifnet.
> 
> The idea is that if_snd_enqueue() tells us whether we grabbed to queue and
> own it or not. If we grabbed it, we go processing it to the end. However,
> we keep accounting on how many packets we processed there. If other
> producer notices that we processed too much, it will preempt the queue.
> 
> Looks like a design that matches your demands. However, extra code needs
> to be put into drivers foo_start() functions, since now we need to disown
> the queue if we stop processing it for some reason different to queue getting
> empty.

I think this patch is not a bad approach.  It resembles the last thing I
posted to net@ except that you have added the burst length cap which is a
nice addition.  Of course, this uses a lock to do so which buf_ring tries to
avoid.

(I also eventually would love to have a way to move the enqueue out of drivers
entirely still where there is a callback for "drain this queue" that only
gets called in the !EBUSY case.  I can't recall if that is compatible with your
stacking approach, but it would make it harder for drivers to get this wrong
if we can avoid duplicating it N times.)

-- 
John Baldwin