From nobody Thu Jun 08 14:00:03 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QcQnz3WPsz4bktN for ; Thu, 8 Jun 2023 14:00:07 +0000 (UTC) (envelope-from rebecca@bsdio.com) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QcQnz1cNhz4LSF for ; Thu, 8 Jun 2023 14:00:07 +0000 (UTC) (envelope-from rebecca@bsdio.com) Authentication-Results: mx1.freebsd.org; none Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 625AF32002D8; Thu, 8 Jun 2023 10:00:05 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 08 Jun 2023 10:00:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdio.com; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1686232804; x=1686319204; bh=iXhv2JvZy4Pqu+Y5tWX+uZWLKZ+Ryh6FjVl 5KCpO/Bw=; b=BBdiM9SeXe0WlZLxVO4uPxVdmLoPH0OxMv9NmuBlZFkAeBPcYi+ MVofmV8I6XxlOJbEfwvLPhTxa7dT0m2lfx/r/8EiG/MbnWcnvL1SL0eP3pvp43wl /+sRBfc/k9dVb79xxKt3+T3Rvbx4SdrGLWMXoP5FWY6Dyo960Hx9Xsf7yTw1z0OW FMixcm/ysaTBOcEAPZrKGgYKHXFq9YuzkOH4b95S5y1cMMOhVGssOw7GVuJ4XrOm Gl+Kv2sRb+Vb+rCxlv/OrjHW+XGtTFaBiUmHXgbDmJObWxJNrliXIBbLwaatiV+3 W6IWOOVCLRqKB8dtR75C7tZb2pkaL3+ymoQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1686232804; x=1686319204; bh=iXhv2JvZy4Pqu+Y5tWX+uZWLKZ+Ryh6FjVl 5KCpO/Bw=; b=SVIyYnSuqA4/HUWTDe6/1aYKz418/oDiT4QV/vKKsg4rFlhmnRo /nhE5UDVKCLBHNEOis1ehQv9LifbBQsbAJrUJb+KVP/t2CDoCAXt0ysYjA7hcJED PmgWqPbmWRlYVF84Zu79TNUO3Vxjc6oKLvwjyZwqOx/qkD9412TSrllUzxNDW1B/ 25AdC59Yerf644/bIs+Qocy3p2LPhqZtdkLYzWk32tpljd2y+AvvUYl5owPciQ4g 1KEfQ+fvXEgd8gB+7712MzdH46GCBZ0KWd3bDqvWJEJNnlcupHfmt5X5QKvzAPON 7R60VS8CZQYwCuQY/Q26AoWGWLc5hB3n96Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrgedtiedgjeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpeftvggs vggttggrucevrhgrnhcuoehrvggsvggttggrsegsshguihhordgtohhmqeenucggtffrrg htthgvrhhnpeegfeegveduheejgfduffefhfehleehiefghfetvdejvdelhfeukefhhfdv geehveenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hrvggsvggttggrsegsshguihhordgtohhm X-ME-Proxy: Feedback-ID: i5b994698:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Jun 2023 10:00:04 -0400 (EDT) Message-ID: Date: Thu, 8 Jun 2023 08:00:03 -0600 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: Seemingly random nvme (nda) write error on new drive (retries exhausted) Content-Language: en-US To: Warner Losh Cc: Tomek CEDRO , FreeBSD CURRENT References: <5b52fc08-fb5a-900e-b98c-817a4ab79846@bsdio.com> From: Rebecca Cran In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4QcQnz1cNhz4LSF X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 6/8/23 05:48, Warner Losh wrote: > > > On Thu, Jun 8, 2023, 4:35 AM Rebecca Cran wrote: > > It's ZFS, using the default options when creating it via the FreeBSD > installer so I presume TRIM is enabled. Without a reliable way to > reproduce the error I'm not sure disabling TRIM will help at the > moment. > > I don't think there's any newer firmware for it. > > > pci gen 4 has a highter error rate so that needs to be managed with > retries.  There's a whole protocol to do that which linux implements. > I suspect the time has come for us to do so too. There's some code > floating around I'll have to track down. Thanks. I dropped the configuration down to PCIe gen 3 and the errors have so far gone away. nda0: nvme version 1.3 x8 (max x8) lanes PCIe Gen3 (max Gen4) link nda1: nvme version 1.3 x4 (max x4) lanes PCIe Gen3 (max Gen4) link -- Rebecca Cran