From nobody Fri Jul 14 18:30:51 2023 X-Original-To: scsi@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R2g614YYHz4mlFf for ; Fri, 14 Jul 2023 18:31:05 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f182.google.com (mail-vk1-f182.google.com [209.85.221.182]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R2g611r2Wz4HZ4 for ; Fri, 14 Jul 2023 18:31:05 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vk1-f182.google.com with SMTP id 71dfb90a1353d-4813422f311so882915e0c.2 for ; Fri, 14 Jul 2023 11:31:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689359464; x=1691951464; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=09uR28IFKkwaSwgVuP9u782VjNZ4Uh/UDLHu/cmNkBU=; b=aJIRPFVarx0tYiXeQgayBQ3EpWBe8XVO9elgxEMKhOBUGClMshKDzecZV2Eaw89Tvo mmjdtJn70L3rKyZ7XIibHpUGanDpthXEXywVVngaHrdKwShRF1KNVve4bsrVkoJz/Oqe 45p9UN5qNC8QVtJorC2QGqhkE5IzlV1MIDJrrLAtdcqo2Qpl2GK1CWInBOx+gu3tVBic FxbGuiokE7S9rjO1nUXS//mT0HSJzG4iGTcyMmNzHjjtIFED7MJwNjH8Q8n13EUHD8tQ 1ENPDZkGljveqtd8oXt4dYUlLTMG4puHk2AeCB480VL++ZTvrzRSFLXUz8xucdnYSCjO 0png== X-Gm-Message-State: ABy/qLYFM73iCeGYkZGoMG1lbNgHkw0LNtPx9ODvSQyzgTez40iVRsJ+ ucnkf/wIIisfp63M9quj+3jmd2iAApgC14Tdaqvo/A71 X-Google-Smtp-Source: APBJJlFO0YYWI9KDT8L2R0WroC0xTCqrHw8xHLyPIznwqopOW6EFcI/fVS+HbDBmO81qV739A7xUAT0W6Z9qPLvtp8E= X-Received: by 2002:a1f:4315:0:b0:481:2ea9:979e with SMTP id q21-20020a1f4315000000b004812ea9979emr3859827vka.5.1689359463628; Fri, 14 Jul 2023 11:31:03 -0700 (PDT) List-Id: SCSI subsystem List-Archive: https://lists.freebsd.org/archives/freebsd-scsi List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Fri, 14 Jul 2023 11:30:51 -0700 Message-ID: Subject: Re: ASC/ASCQ Review To: Warner Losh Cc: scsi@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4R2g611r2Wz4HZ4 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Fri, Jul 14, 2023 at 11:05=E2=80=AFAM Warner Losh wrote= : > > > > On Fri, Jul 14, 2023, 11:12 AM Alan Somers wrote: >> >> On Thu, Jul 13, 2023 at 12:14=E2=80=AFPM Warner Losh wr= ote: >> > >> > Greetings, >> > >> > i've been looking closely at failed drives for $WORK lately. I've noti= ced that a lot of errors that kinda sound like fatal errors have SS_RDEF se= t on them. >> > >> > What's the process for evaluating whether those error codes are worth = retrying. There are several errors that we seem to be seeing (preliminary r= ead of the data) before the drive gives up the ghost altogether. For those = cases, I'd like to post more specific lists. Should I do that here? >> > >> > Independent of that, I may want to have a more aggressive 'fail fast' = policy than is appropriate for my work load (we have a lot of data that's a= copy of a copy of a copy, so if we lose it, we don't care: we'll just dele= te any files we can't read and get on with life, though I know others will = have a more conservative attitude towards data that might be precious and u= nique). I can set the number of retries lower, I can do some other hacks fo= r disks that tell the disk to fail faster, but I think part of the solution= is going to have to be failing for some sense-code/ASC/ASCQ tuples that we= don't want to fail in upstream or the general case. I was thinking of iden= tifying those and creating a 'global quirk table' that gets applied after t= he drive-specific quirk table that would let $WORK override the defaults, w= hile letting others keep the current behavior. IMHO, it would be better to = have these separate rather than in the global data for tracking upstream... >> > >> > Is that clear, or should I give concrete examples? >> > >> > Comments? >> > >> > Warner >> >> Basically, you want to change the retry counts for certain ASC/ASCQ >> codes only, on a site-by-site basis? That sounds reasonable. Would >> it be configurable at runtime or only at build time? > > > I'd like to change the default actions. But maybe we just do that for eve= ryone and assume modern drives... > >> Also, I've been thinking lately that it would be real nice if READ >> UNRECOVERABLE could be translated to EINTEGRITY instead of EIO. That >> would let consumers know that retries are pointless, but that the data >> is probably healable. > > > Unlikely, unless you've tuned things to not try for long at recovery... > > But regardless... do you have a concrete example of a use case? There's a= number of places that map any error to EIO. And I'd like a use case before= we expand the errors the lower layers return... > > Warner My first use-case is a user-space FUSE file system. It only has access to errnos, not ASC/ASCQ codes. If we do as I suggest, then it could heal a READ UNRECOVERABLE by rewriting the sector, whereas other EIO errors aren't likely to be healed that way. My second use-case is ZFS. zfsd treats checksum errors differently from I/O errors. A checksum error normally means that a read returned wrong data. But I think that READ UNRECOVERABLE should also count. After all, that means that the disk's media returned wrong data which was detected by the disk's own EDC/ECC. I've noticed that zfsd seems to fault disks too eagerly when their only problem is READ UNRECOVERABLE errors. Mapping it to EINTEGRITY, or even a new error code, would let zfsd be tuned better.