GDC/libdispatch for kernel

[Robert N. M. Watson]

On 24 Apr 2010, at 20:29, Matthew Fleming wrote:

> I have the need for a simple mechanism to farm off work blocks that will be run on other processors if there is spare compute time, and on the current CPU if there is not any spare time.
> 
> An example of something I want to do is compute an error correcting code and a crc32 checksum on a block of data, potentially while also dealing with system resources so the block can be written to disk.
> 
> I dont want to reinvent the wheel, and the GCD framework looks like it would provide what I want in dispatch groups run on a parallel queue.  However, theres a lot of things supported in GCD that I dont need (now and possibly ever).
> 
> Has anyone done any work to support some of the GCD functions in the kernel?  If not, how useful would other people find such a thing?  In my specific instance, I was thinking along the lines of having the idle thread run a dispatch object off a global queue, setting its priority to the curthread->td_priority of the thread that initiated the dispatch, in order to prevent priority inversions.  I dont want the farmed off work to preempt anything else useful the system may have to do, which is why I was considering the idle threads (or my own threads running at PRI_MAX_IDLE-1 until they have work).
> 
> However, I dont think this implementation is really in the spirit of the rest of the GCD architecture.  So in addition to wondering if there is an existing BSD-licensed code to do such a thing, I also am wondering what direction the community can provide on a few design points to make this useful both for me and for the general community.
> 
> I will be at BSDCan but I need something usable (for me, at least) next week to fix some issues at work.
> 
> Meanwhile, one other question I had was regarding the name of all the dispatch_foo_t objects, which are typedefs of a pointer-to-struct.  This is explicitly frowned upon in style(9), so would it be better to be consistent with libdispatch names or style(9)?  I would tend to assume style(9)
> 

John Baldwin, George Neville-Neil, and I had a similar GCDish conversation while I was in NYC over the weekend. I think this topic calls for a detailed discussion at BSDCan -- there are a number of complicating issues, not least that C Blocks (closures) require C runtime changes and perform memory allocation. When I first got involved in GCD, I commented to Apple that there are marked (but unsurprising) similarities to existing concurrency frameworks we use in the kernel, such as netisr/taskqueue/etc for asynchronous work scheduling. However, the impact of features like Blocks shouldn't be underestimated: they bring significant code simplicity and correctness benefits that are part of what makes the GCD model work so well.

I think for your immediate needs, a refinement of our notion of taskqueue might be the best way forward, though, with a more general discussion of the technologies and approaches in GCD at a later date.

Robert