New ZFS changes in Illumos (13314)

[Martin Matuska]

Illumos has made another change to ZFS, this time dealing with two of
their isuses:
734 taskq_dispatch_prealloc() desired
943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP

New code (changeset 13414):
http://hg.openindiana.org/illumos-gate/rev/b42c1f0432b6

Would be implementing this change to our code of value?

Issue 734:
https://www.illumos.org/issues/734

Description:
We would like an in-kernel interface for non-dynamic taskq's where the
caller supplies the storage for the taskq_ent_t.

The reason for this is to avoid hitting the kmem_alloc()/kmem_free()
calls on very hot code paths. The other motivation (perhaps bigger) is
that if we can avoid the allocation, we can avoid having to deal with
scheduling failures on hot code paths, or in situations where it might
not be easy to recover (e.g. interrupt context) -- by providing the
storage before hand, the taskq_dispatch_prealloc() can degenerate to
mutex_enter, twiddle list pointers, cv_broadcast/signal, mutex_exit.
Easy peasey.

Issue 943:
https://www.illumos.org/issues/943

Description:
While digging through bug reports and such looking for hints toward an
end-user deadlock, I happened upon a report suggesting that ZFS was
calling taskq_dispatch with TQ_SLEEP while in interrupt context. Code
inspection appears to bear this out.

The call chain ends up being: sdintr --> ... --> biodone -->
vdev_disk_io_intr --> zio_interrupt --> zio_taskq_dispatch -->
taskq_dispatch, where zio_taskq_dispatch explicitly ors TQ_SLEEP into
any flags argument:
source:usr/src/uts/common/fs/zfs/zio.c#L1056
<https://www.illumos.org/projects/illumos-gate/repository/entry/usr/src/uts/common/fs/zfs/zio.c#L1056>

-- 
Martin Matuska
FreeBSD committer
http://blog.vx.sk