IO chunking

[Pokala, Ravi]

-----Original Message-----
From: John Baldwin <jhb at freebsd.org>
Date: Tuesday, August 12, 2014 at 9:57 AM
To: "freebsd-hackers at freebsd.org" <freebsd-hackers at freebsd.org>
Cc: Ravi Pokala <rpokala at panasas.com>
Subject: Re: IO chunking

>On Tuesday, August 12, 2014 2:56:26 am Pokala, Ravi wrote:
>> Hi folks,
>> 
>> I'm doing moderately-large block IO (16KB - 1MB) directly against drive
>> devices (i.e. /dev/adaX), and I see that `iostat -d adaX' reports a
>> transaction size of at most 128KB. I believe this is because
>>transactions
>> are limited to at most MAXPHYS bytes (128KB), and requests larger than
>> that are broken into smaller chunks; is that correct? If so, where does
>> that chunking happen? In low-level GEOM code (geom_io.c, geom_dev.c)? In
>> CAM? In the drive device driver? In VFS?
>
>Note that you can increase MAXPHYS (though you will want to ensure your
>storage controller drivers correctly report their maximum supported size
>and don't just hardcode MAXPHYS).

Yeah. For a semi-related issue, we're planning on upping MAXPHYS to 256KB.
That still artificially limits the transaction size, but it's a bit better.

>The limit appears to be throughout the stack, though largely enforced
>at the top (e.g. in physio() before entering GEOM or the b_pages[] array
>in struct buf).

Looking...

sys/kern/kern_physio.c:
101                         /* Don't exceed drivers iosize limit */
102                         if (bp->b_bcount > dev->si_iosize_max)
103                                 bp->b_bcount = dev->si_iosize_max;


Yeah, that's probably what it is. It looks like geom_dev.c::g_dev_taste()
sets si_iosize_max to MAXPHYS, and nothing in the ATA layer changes it.

>Certainly I've seen folks run with MAXPHYS of 512k, but check your
>drivers.

Yeah. We've been running w/ 256KB in another branch for a while, so it
looks like everything we use is okay.

Thanks,

Ravi

>-- 
>John Baldwin