From nobody Tue Oct 15 18:31:42 2024 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XSjNx0mzTz5YpnG for ; Tue, 15 Oct 2024 18:31:45 +0000 (UTC) (envelope-from seneca@vybenetworks.com) Received: from mail.vex.net (mail.vex.net [98.158.132.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XSjNw4sdNz4l75 for ; Tue, 15 Oct 2024 18:31:44 +0000 (UTC) (envelope-from seneca@vybenetworks.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=vybenetworks.com header.s=VEXNET header.b=xegElJAw; spf=pass (mx1.freebsd.org: domain of seneca@vybenetworks.com designates 98.158.132.68 as permitted sender) smtp.mailfrom=seneca@vybenetworks.com; dmarc=pass (policy=none) header.from=vybenetworks.com Received: from webmail.vybenetworks.com (localhost [127.0.0.1]) by mail.vex.net (Postfix) with ESMTP id 56F8C4E9D8 for ; Tue, 15 Oct 2024 14:31:42 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=vybenetworks.com; s=VEXNET; t=1729017102; bh=UZ2qc3sRET/HKAPn1wwMWYBIYxiuTISt95AiPOBRBJg=; h=Date:From:To:Subject:In-Reply-To:References; b=xegElJAwT5X0LhyuwX+2m8ywvcAfiHn/MEVA2NfDfuLX9BtfR4jZDK1TuUPqXy4FC ns08/TDr8KsvuXEzdV6MSrrI2A43wJcf16XdXRajgYI7WGFji6hMG68exqNpQsA7wM fOMAHEMbFgri+qezzl+D2A9EwviwKxbWnyrvVp2g= List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-questions@freebsd.org Sender: owner-freebsd-questions@FreeBSD.org MIME-Version: 1.0 Date: Tue, 15 Oct 2024 14:31:42 -0400 From: seneca@vybenetworks.com To: freebsd-questions@freebsd.org Subject: Re: ZFS Rename rebooting server In-Reply-To: <6250644a-8e4a-4f44-bea5-068be84640c3@druid.net> References: <6250644a-8e4a-4f44-bea5-068be84640c3@druid.net> User-Agent: Vex.Net Webmail/1.6.9 Message-ID: <29467b51cd7c5c28e09bdcc38b52f0de@vybenetworks.com> X-Sender: seneca@vybenetworks.com Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.991]; DMARC_POLICY_ALLOW(-0.50)[vybenetworks.com,none]; R_DKIM_ALLOW(-0.20)[vybenetworks.com:s=VEXNET]; R_SPF_ALLOW(-0.20)[+mx:c]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; FROM_NO_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:19842, ipnet:98.158.132.0/24, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-questions@freebsd.org]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[vybenetworks.com:+] X-Rspamd-Queue-Id: 4XSjNw4sdNz4l75 X-Spamd-Bar: --- On 14.10.2024 17:32, D'Arcy Cain wrote: > I have never seen this before. I am running bhyve on ZFS and I want to > rename the VM. I shut down the server and do "zfs rename old new" and > BOOM! it reboots. No wait, just immediate. In short, that server had a kernel panic that appears to be related to OpenZFS issue #16274: "Kernel oops (null pointer dereference) when renaming a zvol, if snapdev=visible AND the zvol has at least one snapshot". For reference, it currently runs 14.1-RELEASE-p5. > Anyone know how to debug something like this? I started by connecting to the server's built-in remote KVM. From it, I saw, and photographed, the rather distinctive output generated by a kernel panic (a page fault) resulting from a zfs rename attempt. A quick search online using more general terms was not useful, so I then went to OpenZFS' Github repository and searched for issues that included the term 'z_zvol', the name of the panic's current process. Issue #16274 was near the top of the returned list, and the conditions mentioned in the title matched the conditions on the server and zfs rename in question. From the longer comment thread in the issue, I saw a rather similar stack trace to the one from the server (although Linux, not FreeBSD), using the same version of OpenZFS as FreeBSD 14.1, and a confirmation that the issue affected FreeBSD 15. This issue is triggered specifically by trying to rename zvols with snapshots and snapdev=visible. I then set snapdev=hidden on the zvol to be renamed and watched another attempt at renaming it. This time, the zfs rename succeeded with no panic. -- Seneca Cunningham