Re: panic(s) in ZFS on CURRENT

From: Martin Matuska <mm_at_FreeBSD.org>
Date: Fri, 09 Jun 2023 15:39:54 UTC
I will wait with my prepared merge until #14954 gets merged.

On 9. 6. 2023 15:59, Alexander Motin wrote:
> Hi Gleb,
>
> There are two probably related PRs upstream:
> https://github.com/openzfs/zfs/pull/14939
> https://github.com/openzfs/zfs/pull/14954
>
> On 09.06.2023 00:57, Gleb Smirnoff wrote:
>> On Thu, Jun 08, 2023 at 07:56:07PM -0700, Gleb Smirnoff wrote:
>> T> I'm switching to INVARIANTS kernel right now and will see if that 
>> panics earlier.
>>
>> This is what I got with INVARIANTS:
>>
>> panic: VERIFY3(dev->l2ad_hand <= dev->l2ad_evict) failed 
>> (225142071296 <= 225142063104)
>>
>> cpuid = 17
>> time = 1686286015
>> KDB: stack backtrace:
>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame 
>> 0xfffffe0160dcea90
>> kdb_backtrace() at kdb_backtrace+0x46/frame 0xfffffe0160dceb40
>> vpanic() at vpanic+0x21f/frame 0xfffffe0160dcebe0
>> spl_panic() at spl_panic+0x4d/frame 0xfffffe0160dcec60
>> l2arc_write_buffers() at l2arc_write_buffers+0xcda/frame 
>> 0xfffffe0160dcedf0
>> l2arc_feed_thread() at l2arc_feed_thread+0x547/frame 0xfffffe0160dceec0
>> fork_exit() at fork_exit+0x122/frame 0xfffffe0160dcef30
>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0160dcef30
>> --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
>> Uptime: 1m4s
>> Dumping 5473 out of 65308 
>> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>>
>> (kgdb) frame 4
>> #4  0xffffffff804342ea in l2arc_write_buffers 
>> (spa=0xfffffe022e942000, dev=0xfffffe023116a000, target_sz=16777216)
>>      at /usr/src/FreeBSD/sys/contrib/openzfs/module/zfs/arc.c:9445
>> 9445                    ASSERT3U(dev->l2ad_hand, <=, dev->l2ad_evict);
>> (kgdb) p dev
>> $1 = (l2arc_dev_t *) 0xfffffe023116a000
>> (kgdb) p dev->l2ad_hand
>> $2 = 225142071296
>> (kgdb) p dev->l2ad_evict
>> $3 = 225142063104
>> (kgdb) p *dev
>> value of type `l2arc_dev_t' requires 66136 bytes, which is more than 
>> max-value-size
>>
>> Never seen kgdb not being able to print a structure that reported to 
>> be too big.
>>
>