geom <-> cam disk

[Andriy Gapon]

on 26/07/2012 00:14 Scott Long said the following:
> Once the bio is put into the bioq from da_strategy, the CAM scheduler is
> called.  It may or may not wind up calling dastart right away; if the simq or
> devq is frozen, or if the devq has been exhausted, then the io will be
> deferred until later and the call stack will unwind back into g_down.  The
> bioq can therefore accumulate many bio's before being drained.  Draining will
> usually happen from the camisr, at which point you can potentially have i/o
> being initiated from both the camisr and the g_down threads in parallel.  The

Uh-hah.  Thank you for the answer.  I didn't think of the case of
frozen/exhausted queues and also didn't hit in my tests.
Now I am starting to understand the logic in xpt_run_dev_allocq.

BTW, I think that it would be nice if the GEOM work-processing could re-use the
CAM model.
That is, try to execute GEOM bio transformations in the original thread as much
as possible, defer work to the GEOM thread as the last resort.

> monolithic locking in CAM right now prevents this from actually happening,
> though that's a topic that needs to be revisited.


> On Jul 25, 2012, at 1:27 PM, Andriy Gapon wrote:
> 
>> 
>> 
>> Preamble.  I am trying to understand in detail how things work at GEOM <->
>> "CAM disk" boundary.  I am looking at scsi_da and ata_da which seem to be
>> twins in this respect.
>> 
>> I got an impression that the bioq_disksort calls in the strategy methods
>> and the related queues are completely useless in the GEOM single-threaded
>> world. There is only one thread, g_down, that can call a strategy method,
>> the method enqueues a bio, then calls a schedule function and through
>> xpt_schedule the call flow continues to a start method which dequeues the
>> bio and off it goes. I currently can see how a bio queue can accumulate
>> more than one bio.
>> 
>> What am I missing? :-) I will be very glad to learn more about this layer
>> if anyone is willing to educate me. Thank you in advance.
>> 
>> P.S. I wrote a very simple to DTrace script to my "theory" experimentally
>> and my testing with various workloads didn't disprove the theory so far
>> (which doesn't mean that it is correct, of course).
>> 
>> The script: fbt::bioq_disksort:entry /args[0]->queue.tqh_first == 0/ { 
>> @["empty"] = count(); }
>> 
>> fbt::bioq_disksort:entry /args[0]->queue.tqh_first != 0/ { @["non-empty"] =
>> count(); }
>> 
>> It works on all bioq_disksort calls, but I stressing only ada disks at the
>> moment. -- Andriy Gapon
>> 
> 


-- 
Andriy Gapon