Panic on BETA1 in the ZFS subsystem

[Karl Denninger]

On 7/21/2016 07:52, Andriy Gapon wrote:
> On 21/07/2016 15:25, Karl Denninger wrote:
>> The crash occurred during a backup script operating, which is (roughly)
>> the following:
>>
>> zpool import -N backup (mount the pool to copy to)
>>
>> iterate over a list of zfs filesystems and...
>>
>> zfs rename fs at zfs-base fs at zfs-old
>> zfs snapshot fs at zfs-base
>> zfs send -RI fs at zfs-old fs at zfs-base | zfs receive -Fudv backup
>> zfs destroy -vr fs at zfs-old
>>
>> The first filesystem to be done is the rootfs, that is when it panic'd,
>> and from the traceback it appears that the Zio's in there are from the
>> backup volume, so the answer to your question is "yes".
> I think that what happened here was that a quite large number of TRIM
> requests was queued by ZFS before it had a chance to learn that the
> target vdev in the backup pool did not support TRIM.  So, when the the
> first request failed with ENOTSUP the vdev was marked as not supporting
> TRIM.  After that all subsequent requests were failed without sending
> them down the storage stack.  But the way it is done means that all the
> requests were processed by the nested zio_execute() calls on the same
> stack.  And that lead to the stack overflow.
>
> Steve, do you think that this is a correct description of what happened?
>
> The state of the pools that you described below probably contributed to
> the avalanche of TRIMs that caused the problem.
>

The source for the backup a pool that is comprised entirely of SSDs (and
thus supports TRIM), and the target is a pair of spinning rust devices
(which of course do not support TRIM); the incremental receive to that
pool does (of course) remove all the obsolete snapshots.....

What I don't understand however, is why it has been running fine for a
week or so, and why it immediately repeated the panic on a retry attempt
-- or how to prevent it, at least at this point.  I certainly do not
want to leave the pool mounted when not in active backup use.

-- 
Karl Denninger
karl at denninger.net <mailto:karl at denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2996 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20160721/6216b7d4/attachment.bin>