From nobody Mon Feb 28 00:16:53 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E098419DC745 for ; Mon, 28 Feb 2022 00:16:55 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [192.147.25.65]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K6LXl0HFwz57cM; Mon, 28 Feb 2022 00:16:55 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Content-Transfer-Encoding:Content-Type:Message-ID:References: In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=tixOveRqF0vJjItXLlBCa/L+X87u8LqaeaS9Dl1huy0=; b=G7HcT9TVxANOa5I4ule3Kbe+AO LmVvfCah4A9++f6mlXmVNoEIf9MhwF561yljHrMyja+iRIPEUfJ7tei+WcD1DjD5eDFDlCQnoW/+R MUbD50G65VV8kfZ9rHzqCrcW0ow1vb/sBepKs90L7yc8N7iTwFjEqX9D8nR9I5jCNvP/6aO+DM7VT 2q1C+e+rzqkpv+XAc54dCfn6fN2NqHV5ZoDSa+ElIgfzXUtV+280OqB8eGU1YL9Ev864MZxdBY28z /l0tvQc167uIOTawQeODKpb+ZvKmqF9zIKhmVQnu8iFB11optou+XTD38Rx8JHRH73aaVGIO39IG6 kpwDxxiQ==; Received-SPF: pass (thebighonker.lerctr.org: domain of lerctr.org designates 2001:470:1f0f:3ad:bb:dcff:fe50:d900 as permitted sender) client-ip=2001:470:1f0f:3ad:bb:dcff:fe50:d900; envelope-from=ler@lerctr.org; helo=webmail.lerctr.org; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:56160 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1nOTif-000MAK-DU; Sun, 27 Feb 2022 18:16:53 -0600 Received: from 2600:1700:210:b18f:148c:fa9b:bdf:1b29 by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sun, 27 Feb 2022 18:16:53 -0600 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Sun, 27 Feb 2022 18:16:53 -0600 From: Larry Rosenman To: Mark Johnston Cc: Freebsd current Subject: Re: ZFS PANIC: HELP. In-Reply-To: References: <36d3896af19acc4fdd1712822ba9d420@lerctr.org> <9f6a8ad62fc0dbd6f3a19c7d695bd302@lerctr.org> <20220225091120.Horde.76VjoVNtr5BsqAw_5ftjpRZ@webmail.leidinger.net> <04CB94C0-93FE-402B-A24E-1BDAC0D87C5D@lutter.sk> <8df623691922973b5e9b8609d78c4cfa@lerctr.org> <85057850abd46331c95b9a2f3a0c07d8@lerctr.org> <740a16472bb1073174efa709f795896c@lerctr.org> Message-ID: <20d8b59ddfa82275ac771a50921f9300@lerctr.org> X-Sender: ler@lerctr.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4K6LXl0HFwz57cM X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=lerctr.org header.s=ler2019 header.b=G7HcT9TV; dmarc=pass (policy=none) header.from=lerctr.org; spf=pass (mx1.freebsd.org: domain of ler@lerctr.org designates 192.147.25.65 as permitted sender) smtp.mailfrom=ler@lerctr.org X-Spamd-Result: default: False [-2.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[lerctr.org:s=ler2019]; FREEFALL_USER(0.00)[ler]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[lerctr.org:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[lerctr.org,none]; SUBJ_ALL_CAPS(1.20)[16]; MLMMJ_DEST(0.00)[freebsd-current]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8166, ipnet:192.147.25.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 02/27/2022 3:58 pm, Mark Johnston wrote: > On Sun, Feb 27, 2022 at 01:16:44PM -0600, Larry Rosenman wrote: >> On 02/26/2022 11:08 am, Larry Rosenman wrote: >> > On 02/26/2022 10:57 am, Larry Rosenman wrote: >> >> On 02/26/2022 10:37 am, Juraj Lutter wrote: >> >>>> On 26 Feb 2022, at 03:03, Larry Rosenman wrote: >> >>>> I'm running this script: >> >>>> #!/bin/sh >> >>>> for i in $(zfs list -H | awk '{print $1}') >> >>>> do >> >>>> FS=$1 >> >>>> FN=$(echo ${FS} | sed -e s@/@_@g) >> >>>> sudo zfs send -vecLep ${FS}@REPAIR_SNAP | ssh >> >>>> ler@freenas.lerctr.org cat - \> $FN >> >>>> done >> >>>> >> >>>> >> >>>> >> >>> I’d put, like: >> >>> >> >>> echo ${FS} >> >>> >> >>> before “sudo zfs send”, to get at least a bit of a clue on where it >> >>> can get to. >> >>> >> >>> otis >> >>> >> >>> >> >>> — >> >>> Juraj Lutter >> >>> otis@FreeBSD.org >> >> I just looked at the destination to see where it died (it did!) and I >> >> bectl destroy'd the >> >> BE that crashed it, and am running a new scrub -- we'll see whether >> >> that was sufficient. >> >> >> >> Thanks, all! >> > Well, it was NOT sufficient.... More zfs export fun to come :( >> >> I was able to export the rest of the datasets, and re-install >> 14-CURRENT >> from a recent snapshot, and restore the datasets I care about. >> >> I'm now seeing: >> mfi0: IOCTL 0x40086481 not handled >> mfi0: IOCTL 0x40086481 not handled >> mfi0: IOCTL 0x40086481 not handled >> mfi0: IOCTL 0x40086481 not handled >> pid 48 (zpool), jid 0, uid 0: exited on signal 6 >> mfi0: IOCTL 0x40086481 not handled >> mfi0: IOCTL 0x40086481 not handled >> mfi0: IOCTL 0x40086481 not handled >> mfi0: IOCTL 0x40086481 not handled >> pid 54 (zpool), jid 0, uid 0: exited on signal 6 >> >> On boot. Ideas? > > That ioctl is DIOCGMEDIASIZE, i.e., something is asking /dev/mfi0, the > controller device node, about the size of a disk. Presumably this is > the result of some kind of misconfiguration somewhere, and /dev/mfid0 > was meant instead. per advice from markj@ I deleted the /{etc,boot}/zfs/zpool.cache files, and this issue went away. Stale cache files which are no longer needed. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106