taskqueues

[Orit Moskovich]

>From the SMPng Design Document, I read that interrupt threads run at real-time kernel priority.
So wouldn't it be better to define a taskqueue using TASKQUEUE_FAST_DEFINE, but instead of ' taskqueue_start_threads(&taskqueue_foo, 1, PI_NET, "foo taskq"); ', use 
swi_add(NULL, "...", taskqueue_swi_run, NULL, PI_NET,
                     INTR_MPSAFE, &taskqueue_ih))
?

-----Original Message-----
From: John Baldwin [mailto:jhb at freebsd.org] 
Sent: Wednesday, May 29, 2013 09:17 PM
To: freebsd-drivers at freebsd.org
Cc: Orit Moskovich
Subject: Re: taskqueues

On Thursday, May 23, 2013 9:47:14 am Orit Moskovich wrote:
> Hi,
> 
> Can you please specify the difference between interrupt threads and 
> regular
kernel threads in the context of the different default taskqueues?
> Meaning, I saw that the taskqueue taskqueue_thread is actually a 
> kernel
thread running the function taskqueue_thread_loop,
> And the 3 other default taskqueues are working with ithreads.
> 
> Which of the above preempts the other? What should I use if 
> performance is
critical (something equivalent to Linux softirq...)?

ithreads use the most important priority range:

#define	PRI_MIN			(0)		/* Highest priority. */
#define	PRI_MAX			(255)		/* Lowest priority. */

#define	PRI_MIN_ITHD		(PRI_MIN)
#define	PRI_MAX_ITHD		(PRI_MIN_REALTIME - 1)

#define	PI_REALTIME		(PRI_MIN_ITHD + 0)
#define	PI_AV			(PRI_MIN_ITHD + 4)
#define	PI_NET			(PRI_MIN_ITHD + 8)
#define	PI_DISK			(PRI_MIN_ITHD + 12)
#define	PI_TTY			(PRI_MIN_ITHD + 16)
#define	PI_DULL			(PRI_MIN_ITHD + 20)
#define	PI_SOFT			(PRI_MIN_ITHD + 24)
#define	PI_SWI(x)		(PI_SOFT + (x) * RQ_PPQ)

For example, an INTR_TYPE_NET interrupt will generally run in an ithread with the PI_NET priority.  For software interrupt threads the priority is generally PI_SWI(SWI_xxx).  The SWI_xxx constants are:

/*
 * Software interrupt numbers in priority order.  The priority determines
 * the priority of the corresponding interrupt thread.
 */
#define	SWI_TTY		0
#define	SWI_NET		1
#define	SWI_CAMBIO	2
#define	SWI_VM		3
#define	SWI_CLOCK	4
#define	SWI_TQ_FAST	5
#define	SWI_TQ		6
#define	SWI_TQ_GIANT	6

For taskqueues you have three global taskqueues to choose from:

taskqueue_fast: runs at PI_SWI(SWI_TQ_FAST) = 44
taskqueue_swi: runs at PI_SWI(SWI_TQ) = 48
taskqueue_thread: runs at PWAIT = 108

However, you can create your own thread-backed taskqueue that can run at whatever priority you choose.  There is nothing really special about having the taskqueue be an SWI vs a normal thread.  If you wanted a taskqueue that ran at PI_NET you could do:

TASKQUEUE_DEFINE(foo, taskqueue_thread_enqueue, &taskqueue_foo,
    taskqueue_start_threads(&taskqueue_foo, 1, PI_NET, "foo taskq");

If you needed to queue tasks from an interrupt filter you would want to use a fast taskqueue instead:

TASKQUEUE_FAST_DEFINE(< same args as above >)

--
John Baldwin