ZFS UNMAP performance

[Eric van Gyzen]

On 03/11/2014 15:47, Danny Schales wrote:
> On 03/11/2014 11:08, Danny Schales wrote:
>> On 03/11/2014 10:48, Tom Evans wrote:
>>> On Tue, Mar 11, 2014 at 3:28 PM, Danny Schales <dan at latech.edu> wrote:
>>>> I'm seeing very slow performance with certain operations on a ZFS
>>>> filesystem built on ISCSI LUN's on a 10.0 system (new ISCSI
>>>> implementation).  The issue appears to be with BIO_DELETE operations.
>>>> Monitoring the system with gstat shows expected times for read and write
>>>> operations, but deletes are in the multiple hundreds of milliseconds
>>>> under normal operation.  Destroying a snapshot sends the times to
>>>> astronomical levels.  sysctl says the system is using UNMAP for deletes:
>>>>
>>>> kern.cam.da.0.delete_method: UNMAP
>>>>
>>>> I searched and found where Oracle issued a performance alert for Solaris
>>>> 11.1 where ZFS using UNMAP was in use.  Here's a link to a blog
>>>> discussing it:
>>>>
>>>> http://schalwad.blogspot.com/2013/12/solaris-111-zfs-write-performance.html
>>>>
>>>>
>>>> Is FreeBSD also impacted?  If so, is there a fix or a workaround?
>> The backend is an ISCSI LUN from a SAN (Dell Compellent).  From
>> research, it seems the SAN *does* support SCSI UNMAP requests for
>> deletes, but the performance is horrible.  I don't know if this is a
>> FreeBSD issue or a Compellent issue.  I haven't seen the problem with
>> other devices, but I don't think anything else is using UNMAP (yet).
>>
>> Danny
>>
>>
> Replying to myself...I note that the system is reporting that TRIM is
> being used.  Is this normal for non-SSD systems?  There *is* SSD in the
> system, but I'm pretty sure the system can't tell it's SSD (it's hidden
> behind a Dell PERC card).  The number of trim.successes is roughly
> equivalent to the number of deletes reported by gstat for the ISCSI LUN
> devices.  Should the system be using TRIM for ISCSI LUNs?

Sure, if the LUN (i.e. the storage controller) reports that it supports
TRIM/UNMAP.  Note that this is completely unrelated to the type of disks
that provide the LUN's backing store.

> kstat.zfs.misc.zio_trim.bytes: 232845656064
> kstat.zfs.misc.zio_trim.success: 30810983
> kstat.zfs.misc.zio_trim.unsupported: 809
> kstat.zfs.misc.zio_trim.failed: 0
>
> Danny
>