How to send 1MB I/O size in a single I/O request without split
Scott Long
scott4long at yahoo.com
Thu Jan 29 20:17:16 UTC 2015
> On Jan 29, 2015, at 11:56 AM, Sibananda Sahu <sibananda.sahu at avagotech.com> wrote:
>
> Hi All,
>
>
>
> Recently we have added large I/O size of 1MB in our LSI controller and for
> that we have implemented the same in driver.
>
> But I have observed that the large I/O that an application is able to send
> is 128KB in one I/O request.
>
>
>
> I used the following command to send 1MB I/O:
>
> # ddpt if=/dev/da0 of=/dev/zero count=1 bs=1M
>
>
>
> But I have observed that the number of scatter gather elements per I/O
> request always comes 1 and the I/O length comes as 128KB(max).
>
> […]
>
>
> So my primary questions are:
>
> - How can I send a large I/O size of 1MB in a single I/O request
> without any split?
You answered this question already, you must redefine MAXPHYS. This can be done via a kernel compile option, and many users and companies already know how to do this. I plan to start a discussion on increasing the default size.
>
> - Why I am getting always 1 scatter gather element?
>
You are likely getting an allocation that is physically contiguous. This is especially true since you are using ‘dd’ from userland, and the memory allocator in userland tries to useon 2MB superpages for allocations. The busdma API will see that the allocation is contiguous and attempt to merge the contiguous segments. This is usually desirable since few segments reduces processing overhead in the driver and the hardware.
> - How can I get more sge count in an I/O request?
>
If you want to test multiple segments, I suggest leaving you system running for a long time with multiple processes freeing and allocating memory so that the system becomes fragmented. You can also modify the bus_dma_tag in your driver to specify a maximum segment size of 4k instead of (presumably in your case) something larger. That will force busdma to stop merging adjacent segments.
Scott
More information about the freebsd-scsi
mailing list