From nobody Wed Jul 06 04:42:54 2022 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9B5911D0B4C4 for ; Wed, 6 Jul 2022 04:42:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ld6Nb2WXnz3qM0 for ; Wed, 6 Jul 2022 04:42:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 36724138C2 for ; Wed, 6 Jul 2022 04:42:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 2664gtFY068871 for ; Wed, 6 Jul 2022 04:42:55 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 2664gtuo068870 for fs@FreeBSD.org; Wed, 6 Jul 2022 04:42:55 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 264141] nvme(4): Heavy load to SSD wedges 13.1 system: Controller in fatal status, resetting ... Resetting controller due to a timeout and possible hot unplug. Date: Wed, 06 Jul 2022 04:42:54 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.1-RELEASE X-Bugzilla-Keywords: needs-qa, regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: dgilbert@eicat.ca X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1657082575; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qH81zIQ+1MAW8IjA5vjVhbPKeSbQ9pMCmx+Btb1pykA=; b=TRmwIGtlPQ6w7G2AaqpYAQ9wIW5BILsflmZ138w+YSdfzwu8fh3z4rK7Ip8DvBeqTL3rJG p9Jw4kgoeKUcj5UdSR9Csx2F0kWlDhOTUKsg0vaiAA+s3QMKRjE4geEy9owAj/3VhTK5ED Y/nhtFrnGSt2GeDuYBXW37CsLtZKaiCGeZqT6D3W3JzT3LqanUzlLgYVXLDJ5irdhGhJui DfUUoAzMnZn/SmaS3FMBMO3Thf0Hv/22yNA5rx1QIqah/5EknVCIEyaijBd0FlObgs9/Pj vp6oxdvBvKJgWfE93qOpDzBZke7ie3lUeBDumUT5AuRw+YyQpDSv3XPAkzfugg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1657082575; a=rsa-sha256; cv=none; b=aIFb0rtlcVY80MzN6Vw0rSFzstwRt+hjOvWMP/eHWY+wydHpCrRUWrqA40lPBM4g6u8LJC oLeZTheHOIUMb7Ve7LLrCTeyLA4g/m+iGkgqLX+Y9S/wnoJv5/FsRDF9zW/bj5sUw3F0lW HVQ9/IxoTUEhw/XR9p++7uBM10dBq2URC+K2eaNnmuAIVptDDAC8aSm2982kUqB8Adg7TY sf80yPehtxuMXuKNnmPX2muHT4dU/jaYrzDGUo29IeSRUDCMy7Ta11VeoW7pINARFhBTpJ A4LIn1i2cQuKxq+a9lVw0HGMaLR6q1uwZhcmq2aLbkm+ji9aaQWk9gMP2JgjJA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D264141 --- Comment #24 from dgilbert@eicat.ca --- (In reply to Warner Losh from comment #23) ... so my messages file back from yesterday says: Jun 15 22:46:17 vr kernel: nvme0: Allocated 200MB host memory buffer ... the trouble starts with: Jun 17 04:28:00 vr kernel: nvme0: RECOVERY_START 459377299057992 vs 459376982792873 Jun 17 04:28:00 vr kernel: nvme0: RECOVERY_START 459377569149703 vs 459376982792873 Jun 17 04:28:00 vr kernel: nvme0: RECOVERY_START 459378788907460 vs 459376987085573 Jun 17 04:28:00 vr kernel: nvme0: RECOVERY_START 459378788907460 vs 459376995708726 Jun 17 04:28:00 vr kernel: nvme0: RECOVERY_START 459379058646386 vs 459376995708726 Jun 17 04:28:00 vr kernel: nvme0: Controller in fatal status, resetting Jun 17 04:28:00 vr kernel: nvme0: Resetting controller due to a timeout and possible hot unplug. Jun 17 04:28:00 vr kernel: nvme0: RECOVERY_WAITING Jun 17 04:28:00 vr kernel: nvme0: resetting controller Jun 17 04:28:00 vr kernel: nvme0: waiting Jun 17 04:28:00 vr kernel: nvme0: failing outstanding i/o Jun 17 04:28:00 vr kernel: nvme0: READ sqid:1 cid:118 nsid:1 lba:2985560513 len:256 Jun 17 04:28:00 vr kernel: nvme0: ABORTED - BY REQUEST (00/07) sqid:1 cid:1= 18 cdw0:0 Jun 17 04:28:00 vr kernel: nvme0: failing outstanding i/o ... which repeats ad-nausium. At least in my case, this was a zfs-cache drive ... so I have a log. ... but sadly, I returned the drive already. I can't afford a $300 non-wor= king bit of hardware. Had to return and order a new one. I can tell you that another NVMe in it's place does not _ever_ do this. An= d I can detail that I RMA'd the first one that did this, and the replacement did the same thing. --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.=