Fwd: Re: Strange ZFS performance

[Mikle Krutov]

On Mon, Apr 05, 2010 at 12:24:15PM -0500, Bob Friesenhahn wrote:
> On Mon, 5 Apr 2010, Mikle Krutov wrote:
> > Any information i could provide to help us know what's the source of the problem?
> 
> The svc_t value I saw posted in one of your mails was outrageously 
> large.  I suggest running
> 
>    iostat -x 30
> 
> while doing long-duration write and see what the actual values are. 
> It seems likely that you are overrunning what your controller or disk 
> is capable of handling and this is creating an I/O backlog which is 
> far greater than what it would be if zfs had queued fewer 
> simultaneous requests.
> 
> It may be that tuning zfs_vdev_max_pending to a small value may help 
> keep your controller from being overloaded.
> 
> Bob
> --
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
With copeing from one disk to another (e.g. reading from ad12), svc_t is 
pretty large (2.0-3.5 on target disk v.s. 11 in the beginning - up to 42 
to the end on ad12).
For writing it is ~20 in the beginning, and pretty fast it enlarges to 200 

Also, if it was the overrun of hdd/controller capability, by my mind,  
io should have been very slow for some amount of time, while zfs is flushing
the data from mem to the hdd, and then it should go up to normal 60MB/s.
It does not happen, e.g. if i'm trying to write something just after 
reboot with no data to be flushed from my mem speed is already 2-6MB/s
But still, i've tried to set vfs.zfs.vdev.max_pending to 8 and 4, it did not help.
-- 
Wbr,
Krutov Mikle