From nobody Sun May 22 05:50:25 2022 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 04E3F1B3F910 for ; Sun, 22 May 2022 05:50:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4L5V1F1Xm9z4VqP for ; Sun, 22 May 2022 05:50:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0F16F15B36 for ; Sun, 22 May 2022 05:50:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 24M5oPJW002711 for ; Sun, 22 May 2022 05:50:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 24M5oPwp002710 for fs@FreeBSD.org; Sun, 22 May 2022 05:50:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 264141] nvme(4): Heavy load to SSD wedges 13.1 system: Controller in fatal status, resetting ... Resetting controller due to a timeout and possible hot unplug. Date: Sun, 22 May 2022 05:50:25 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.1-RELEASE X-Bugzilla-Keywords: needs-qa, regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: crb@ChrisBowman.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653198625; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HRpGUWsfIjnJj3DarTHMjnljd2w9MZAZwVv8vAwxp18=; b=TceYOb4AqZy0N8n5Ern9UAfNvc4leSBMbCbMexXDKTYe+KAUz5o8TpwSs/a6a9ZYHiIA/Q WMFWFAvKjgihC6ZeP6Ulnc4ZLPxwq8u9lQUTjvw8zs+a8X5qMs+zYfgkD7jDgdXP9DGU23 4jk9K1A/elxlX8F/CjehHzqDOQ02cb/riKoCc4uiHgvI7keoO40KqWMeYzbp7VaakzPRLT qvtvYwC3Yb+aI4OV+GvIFrbmN9o4RRtoy3oE6Fxf8I8yTUfBcIxwa1mrzHvCpXwVr6hQ0Q dXHvF2VfGMLEQLoUHVYQgpeGdon6v/gUNzgwppyF0oBv4tXDI1nHUnDyFllI9Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1653198625; a=rsa-sha256; cv=none; b=CBnvLJV3UAP+8213X+OCfz8EJ6ALu/6MsKe4e1kymQ6dC84INjKOdjQ+I+s8VIoFWqZWH+ FZQwl0XzprDrDYR0FuS5huBpYRCDgxL2wod8eJ/4br7JQpBq+nCSoMfUeJoNpewArJ0PcW Wjc0RjkjxknQQqUeP/h+YWk0g/6UlLZ1VDmP3X8XAt7Iv0S0Bcxhx751Kibxb6Z2IxlH1S RjPUxQUScT2UWDbKiY1JDT5xUiANwlU+/YEijhFolhYEe/+dDVV52pr6FzurtqZ5xvWkvB 0RBWf49jnEz9vPy9jTzIC+gU+pfPwJ2MQo9HZZMvJnHof9fVgqfIFoq0Ty1aMQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D264141 --- Comment #8 from crb --- Replacing nvme with nda results in similar looking messages from both nvme0= and nda0 (theses didn't show up in a remote ssh session so that I could cut and paste them). I don't think the cards get to hot. The machine has 3 fans that spin up wi= th cpu temperature and as I mentioned earlier the card has a heat sync. When I link while building world with 32 jobs I do hear the fans ramp ever so slig= htly but mostly they're quiet. I doubt it's cabling as these SSDs were directly inserted in to an M2 slot = and I seated the last one securely a few days ago. It could be power, this is a bit of a hacked system (I gutted a Sun Ultra 40 and replaced the contents with this reusing the power supply) but I don't h= ave a way to eliminate power as a possibility right now. Theoretically this sy= stem should be able to deliver 1000W and I only have the motherboard, processor,= 64 G memory, the SSD, 2 ethernet cards (one a Mellanox CX3 using fiber) and 6 spinning drives which are basically quiet. Power seems unlikely as the sys= tem seems otherwise rock solid with load except when hitting the SSD hard. This (unfortunately) seems to be completely repeatable now simple by copyin= g a couple of repo over 10G ether from a remote nfs machine to the local SSD wh= ile the machine is otherwise completely idle. --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.=