From nobody Sat Jan 14 14:57:04 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NvLwt1LN4z2r0Kg for ; Sat, 14 Jan 2023 14:57:18 +0000 (UTC) (envelope-from milkyindia@gmail.com) Received: from mail-vk1-xa34.google.com (mail-vk1-xa34.google.com [IPv6:2607:f8b0:4864:20::a34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NvLws6csXz3QRj for ; Sat, 14 Jan 2023 14:57:17 +0000 (UTC) (envelope-from milkyindia@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vk1-xa34.google.com with SMTP id w72so11474339vkw.7 for ; Sat, 14 Jan 2023 06:57:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ibTz7qOAjbmebXXdq0uSqAp9so4hlHaJpZY2HRGepic=; b=FsffKN+h3l+Dc8+HoKPbXXtJ64NWSTAdNxsgULyb0ztk9TGpF6iYYatEJiBjjB3j+b XHnMJUlj+pEgCjdUWnFAoYNhs28CyGjy2T0mq0h4U9bnOhXXJNoWUrFHozR8McnhEoiV WgdULTQEJq3pyKpm+84dhu/5AJGwDK7YbffTPnxfRWLdtouKNFNkvE063zy5e+AVygQS 5CAy4r3+z+A5asmVGDUmeRmT0fc5DMAOY7oWDR8XvS/VvUYCcur8fodlYjbNvz6Neo2V d5l/hKzftvRGAcw6dm4vwD3wVPWcfqBkhn6yge5m7X8HE9mY2s5zA9E/xc/XPvUCxJ/u cZNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ibTz7qOAjbmebXXdq0uSqAp9so4hlHaJpZY2HRGepic=; b=U1bgk8qz3l8+2rII7XtxJcKlPX78e08HEFDRAlVfO92qybt7DRr4cWtNTs3vYPvBoC +b0ya9IywWeVIZo7U52QHxZK/yY8P/OqF5lvxVAIcun2k6IWTLp1JT6rv5cSW7rqaR6x qvNPDUWqcxHOeR3Ceg7ltMdLWNMuuIRNNIkG7YmRmPuK7cZB8HTPuV0+F8k5Tb2H9hdw lxti+KvqSapPvfGMP6l5z3G2veq3ZKN9NzMKTM0s8vXb3Wp1VPQ8oygLMipFBSzwDaN+ Sqnkig42LFnWq/sHiXiZqSbmrlLfzMyfx8UmCB1OFqK0Z+X0jBNUPLf1kaR57TruzJtQ 6emw== X-Gm-Message-State: AFqh2kpODPPsnGhv85KwrNRusAXEO2XTHKjhxifcLXiJ/4Fjkmyh4YtE oplyHqIAazUDKQhdmnaejVPtmjzwvSLPda+atclV/HYdZBY= X-Google-Smtp-Source: AMrXdXtpp61reVo2fhj4SL8NIjVrBbpVjJgm8Wah4x5ZtId+X+qChA1q1+fPCIUadQQXX7C4F1Y2SZEUGzf5a6bJ9bI= X-Received: by 2002:a05:6122:1689:b0:3c5:db35:9288 with SMTP id 9-20020a056122168900b003c5db359288mr10906320vkl.32.1673708236780; Sat, 14 Jan 2023 06:57:16 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: milky india Date: Sat, 14 Jan 2023 18:57:04 +0400 Message-ID: Subject: Re: ZFS checksum error on 2 disks of mirror To: freebsd@vanderzwan.org Cc: freebsd-fs Content-Type: multipart/alternative; boundary="000000000000e939a705f23a901a" X-Rspamd-Queue-Id: 4NvLws6csXz3QRj X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000e939a705f23a901a Content-Type: text/plain; charset="UTF-8" > Output of zpool status -v gives no read/write/cksum errors but lists one file with an error. Had faced a similar issue, when I tried to delete the file the error still persisted, although realised it after a few shutdown cycles >After running a scrub on the pool all seems to be well, no more files with errors. Please monitor if the error shows up again sometime soon. While I don't know what the issue is but zfs error no 97 seems like a serious bug. Is this a similar issue for which PR is open? https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268333 On Fri, Jan 13, 2023, 19:35 wrote: > Hi, > I noticed zpool status gave an error for one of my pools. > Looking back in the logs I found thus: > > Dec 24 00:58:39 freebsd ZFS[40537]: pool I/O failure, zpool=backuppool > error=97 > Dec 24 00:58:39 freebsd ZFS[40541]: checksum mismatch, zpool=backuppool > path=/dev/gpt/VGJL4JYGp2 offset=1634427084800 size=53248 > Dec 24 00:58:39 freebsd ZFS[40545]: checksum mismatch, zpool=backuppool > path=/dev/gpt/VGJKNA9Gp2 offset=1634427084800 size=53248 > > These are 2 WD Red Plus 8TB drives (same age, same firmware, attached to > same controller). > > Looking back in the logs I found this occurred earlier without me noticing: > > Aug 8 03:17:56 freebsd ZFS[12328]: pool I/O failure, zpool=backuppool > error=97 > Aug 8 03:17:56 freebsd ZFS[12332]: checksum mismatch, zpool=backuppool > path=/dev/gpt/VGJL4JYGp2 offset=4056214130688 size=131072 > Aug 8 03:17:56 freebsd ZFS[12336]: checksum mismatch, zpool=backuppool > path=/dev/gpt/VGJKNA9Gp2 offset=4056214130688 size=131072 > Aug 8 13:37:26 freebsd ZFS[22317]: pool I/O failure, zpool=backuppool > error=97 > Aug 8 13:37:26 freebsd ZFS[22321]: checksum mismatch, zpool=backuppool > path=/dev/gpt/VGJKNA9Gp2 offset=4056214130688 size=131072 > Aug 8 13:37:26 freebsd ZFS[22325]: checksum mismatch, zpool=backuppool > path=/dev/gpt/VGJL4JYGp2 offset=4056214130688 size=131072 > Aug 8 15:37:44 freebsd ZFS[24704]: pool I/O failure, zpool=backuppool > error=97 > Aug 8 15:37:44 freebsd ZFS[24708]: checksum mismatch, zpool=backuppool > path=/dev/gpt/VGJL4JYGp2 offset=4056214130688 size=131072 > Aug 8 15:37:44 freebsd ZFS[24712]: checksum mismatch, zpool=backuppool > path=/dev/gpt/VGJKNA9Gp2 offset=4056214130688 size=131072 > > Output of zpool status -v gives no read/write/cksum errors but lists one > file with an error. > > After running a scrub on the pool all seems to be well, no more files with > errors. > > System is a homebuilt with Asrock Rack C2550 board with 16 GB of ECC RAM > Any idea how I could get checksum errors on the identical block of 2 disks > in a mirror ? > > Regards, > Paul > --000000000000e939a705f23a901a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
>=C2=A0Output of zpool status -v gives no read/write/cksu= m errors =C2=A0but lists one file with an error.
Had faced a similar issue, when I tried to delete the file the error st= ill persisted, although realised it after a few shutdown cycles

>After running a scrub on the po= ol all seems to be well, no more files with errors.
= Please monitor if the error shows up again sometime soon. While I don't= know what the issue is but zfs error no 97 seems like a serious bug.=C2=A0=

Is this a similar issue= for which PR is open? https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D2683= 33=C2=A0

On Fri, Jan 13, 2023, 19:35 <= ;freebsd@vanderzwan.org> w= rote:
Hi,
I noticed zpool s= tatus gave an error for one of my pools.
Loo= king back in the logs I found thus:
Dec 24 00:58:39 freebsd ZFS[40537]: pool I/O failure= , zpool=3Dbackuppool error=3D97
Dec 24 00:58= :39 freebsd ZFS[40541]: checksum mismatch, zpool=3Dbackuppool path=3D/dev/g= pt/VGJL4JYGp2 offset=3D1634427084800 size=3D53248
Dec 24 00:58:39 freebsd ZFS[40545]: checksum mismatch, zpool=3Dbackup= pool path=3D/dev/gpt/VGJKNA9Gp2 offset=3D1634427084800 size=3D53248<= br style=3D"color:rgb(0,0,0);font-family:Monaco;font-size:12px;font-style:n= ormal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-a= lign:start;text-indent:0px;text-transform:none;white-space:normal;word-spac= ing:0px;text-decoration:none">
These are 2 WD Red P= lus 8TB drives (same age, same firmware, attached to same controller).

Looking back in t= he logs I found this occurred earlier without me noticing:

Aug =C2=A08 03:17:56 freebsd= ZFS[12328]: pool I/O failure, zpool=3Dbackuppool error=3D97
Aug =C2=A08 03:17:56 freebsd ZFS[12332]: checksum mismatch= , zpool=3Dbackuppool path=3D/dev/gpt/VGJL4JYGp2 offset=3D4056214130688 size= =3D131072
Aug =C2=A08 03:17:56 freebsd ZFS[1= 2336]: checksum mismatch, zpool=3Dbackuppool path=3D/dev/gpt/VGJKNA9Gp2 off= set=3D4056214130688 size=3D131072
Aug =C2=A0= 8 13:37:26 freebsd ZFS[22317]: pool I/O failure, zpool=3Dbackuppool error= =3D97
Aug =C2=A08 13:37:26 freebsd ZFS[22321= ]: checksum mismatch, zpool=3Dbackuppool path=3D/dev/gpt/VGJKNA9Gp2 offset= =3D4056214130688 size=3D131072
Aug =C2=A08 1= 3:37:26 freebsd ZFS[22325]: checksum mismatch, zpool=3Dbackuppool path=3D/d= ev/gpt/VGJL4JYGp2 offset=3D4056214130688 size=3D131072
Aug =C2=A08 15:37:44 freebsd ZFS[24704]: pool I/O failure, zpool= =3Dbackuppool error=3D97
Aug =C2=A08 15:37= :44 freebsd ZFS[24708]: checksum mismatch, zpool=3Dbackuppool path=3D/dev/g= pt/VGJL4JYGp2 offset=3D4056214130688 size=3D131072
Aug =C2=A08 15:37:44 freebsd ZFS[24712]: checksum mismatch, zpool=3D= backuppool path=3D/dev/gpt/VGJKNA9Gp2 offset=3D4056214130688 size=3D131072<= /span>

Output of zpo= ol status -v gives no read/write/cksum errors =C2=A0but lists one file with= an error.

Af= ter running a scrub on the pool all seems to be well, no more files with er= rors.

<= span style=3D"color:rgb(0,0,0);font-family:Monaco;font-size:12px;font-style= :normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text= -align:start;text-indent:0px;text-transform:none;white-space:normal;word-sp= acing:0px;text-decoration:none;display:inline!important;float:none">System = is a homebuilt with Asrock Rack C2550 board with 16 GB of ECC RAMAny idea how I could get checksum errors on the ident= ical block of 2 disks in a mirror ?
Regards,
Paul
--000000000000e939a705f23a901a--