How to send 1MB I/O size in a single I/O request without split

[Scott Long]

> On Jan 30, 2015, at 2:41 AM, Sibananda Sahu <sibananda.sahu at avagotech.com> wrote:
> 
> Hey Scoot,
> 
> Thanks very much for your reply.
> 
>>> -          How can I get more sge count in an I/O request?
>>> 
> 
>> If you want to test multiple segments, I suggest leaving you system running
>> for a long time with multiple processes freeing and allocating memory so
>> that the system becomes >fragmented.  You can also modify the bus_dma_tag
>> in your driver to specify a maximum segment size of 4k instead of
>> (presumably in your case) something larger.  That will force >busdma to
>> stop merging adjacent segments.
> 
> As you have said:
> 
> "The busdma API will see that the allocation is contiguous and attempt to
> merge the contiguous segments.  This is usually desirable since few segments
> reduces processing overhead in the driver and the hardware."
> 
> If I will modify the bus_dma_tag in our driver to have a max segment of 4K
> then what is the performance impact?
> 

Most drivers have a for() loop that iterates through the list of segments from
busdma and stores them into SGE elements for the hardware to use.  Obviously,
fewer segments means fewer trips through this loop.  It also means a smaller
SGE list, which means less command data transferred over the PCI bus.  Though
I cannot speak for the architecture details of your hardware, often times the same
optimizations apply to the hardware internally.  With modern storage controllers
that are doing millions of i/o’s per second, these small improvements can add up.

Scott