From nobody Fri Jun 09 15:39:54 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qd4yh4mwJz4bF6N for ; Fri, 9 Jun 2023 15:39:56 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qd4yh4NB0z3MRs; Fri, 9 Jun 2023 15:39:56 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from sslproxy05.your-server.de ([78.46.172.2]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1q7eDT-000PI7-KN; Fri, 09 Jun 2023 17:39:55 +0200 Received: from [188.167.171.2] (helo=[10.0.9.122]) by sslproxy05.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q7eDT-000ALC-DK; Fri, 09 Jun 2023 17:39:55 +0200 Message-ID: <397cda46-d02f-5aff-04b1-61fc497ba775@FreeBSD.org> Date: Fri, 9 Jun 2023 17:39:54 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: panic(s) in ZFS on CURRENT Content-Language: en-US To: Alexander Motin , Gleb Smirnoff , current@freebsd.org References: <07c3ec8e-5089-d171-0c8d-1916249f9fd7@FreeBSD.org> From: Martin Matuska In-Reply-To: <07c3ec8e-5089-d171-0c8d-1916249f9fd7@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Authenticated-Sender: martin@matuska.de X-Virus-Scanned: Clear (ClamAV 0.103.8/26934/Fri Jun 9 09:25:21 2023) X-Rspamd-Queue-Id: 4Qd4yh4NB0z3MRs X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N I will wait with my prepared merge until #14954 gets merged. On 9. 6. 2023 15:59, Alexander Motin wrote: > Hi Gleb, > > There are two probably related PRs upstream: > https://github.com/openzfs/zfs/pull/14939 > https://github.com/openzfs/zfs/pull/14954 > > On 09.06.2023 00:57, Gleb Smirnoff wrote: >> On Thu, Jun 08, 2023 at 07:56:07PM -0700, Gleb Smirnoff wrote: >> T> I'm switching to INVARIANTS kernel right now and will see if that >> panics earlier. >> >> This is what I got with INVARIANTS: >> >> panic: VERIFY3(dev->l2ad_hand <= dev->l2ad_evict) failed >> (225142071296 <= 225142063104) >> >> cpuid = 17 >> time = 1686286015 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame >> 0xfffffe0160dcea90 >> kdb_backtrace() at kdb_backtrace+0x46/frame 0xfffffe0160dceb40 >> vpanic() at vpanic+0x21f/frame 0xfffffe0160dcebe0 >> spl_panic() at spl_panic+0x4d/frame 0xfffffe0160dcec60 >> l2arc_write_buffers() at l2arc_write_buffers+0xcda/frame >> 0xfffffe0160dcedf0 >> l2arc_feed_thread() at l2arc_feed_thread+0x547/frame 0xfffffe0160dceec0 >> fork_exit() at fork_exit+0x122/frame 0xfffffe0160dcef30 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0160dcef30 >> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >> Uptime: 1m4s >> Dumping 5473 out of 65308 >> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% >> >> (kgdb) frame 4 >> #4  0xffffffff804342ea in l2arc_write_buffers >> (spa=0xfffffe022e942000, dev=0xfffffe023116a000, target_sz=16777216) >>      at /usr/src/FreeBSD/sys/contrib/openzfs/module/zfs/arc.c:9445 >> 9445                    ASSERT3U(dev->l2ad_hand, <=, dev->l2ad_evict); >> (kgdb) p dev >> $1 = (l2arc_dev_t *) 0xfffffe023116a000 >> (kgdb) p dev->l2ad_hand >> $2 = 225142071296 >> (kgdb) p dev->l2ad_evict >> $3 = 225142063104 >> (kgdb) p *dev >> value of type `l2arc_dev_t' requires 66136 bytes, which is more than >> max-value-size >> >> Never seen kgdb not being able to print a structure that reported to >> be too big. >> >