From nobody Sat Jul 20 02:59:11 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WQrq55S5Pz5QxxK; Sat, 20 Jul 2024 02:59:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WQrq442V1z438m; Sat, 20 Jul 2024 02:59:12 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721444352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=prHUlp6TLCbIick2noDkmoC5LdAv4w8/9m1ENF+ZdkM=; b=bOjSw1v8IzfaroDNHRlA/sEGk7x++5OoAsjeUs26GZ+xxoDln7iLlQ5aLGlk8GEQAYzFLL ddJz3unyxsk1Y7atLhVIWNXsQgdH5atsu/ZjbUXjdFrr8cEQapqjam7fij7FZNsTt+PZOF 5hTW1u+v00CSxbQqWcp+d6rOEmGiZwXCAKAIf3QbBlTVYz+UxR5OqhlbLr/Eed+P4xGpP3 01U2dh+sW6Ya57Bu1Qv4tgRJ2ywjr4gp2hClV4Gj+ItIGqpur0dhyxCiFRR5RcBAysKSko kLG4bzHfVkhrANmwUvN3tEd8gFdXWmNyK89+pj10/Pi9KDAgwD1h/rv2WjzM4Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721444352; a=rsa-sha256; cv=none; b=P//zVt8AZu8XFAuwOPDbt3qeZfOCC67gBU7lEGyfbA3Vsn1cYrZa3AW9Z/nd67kb60fNQA Fa9oaFLBz4doaPJxLaOlhnBBAJMAumaTuQ7J85nYB5oQKnjWfFaHHNkDIBZd2P4edQqi8t /zdZE69YEEQkaW5Oy5GgeEsO27rat1K0+xZeomnXp64bGWzb4bqwXPkX/99symIOYKqBA5 L6Y0G+g7ADcrEjcarlPG85WUpAzB/8WQddNwi9VAMRr3EWAyB0DkgPeC+WmtUrtNsQVdak 7SNCfACbI+kdcbT/a/UvMJhdmkG+4CWjoioMxnzhSj8Rth8av+vkg5zqM1N/LA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721444352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=prHUlp6TLCbIick2noDkmoC5LdAv4w8/9m1ENF+ZdkM=; b=glqPw559ItTsLqniQjUL90PXeITuM+ABygR971xrZkpCsD3cAfJi2SyG62UTSg5jvkj3oI uA8HXK8FvTlfCPm3R723MnhLcHb72VslnyuKodJ1fNKSdu4+A2vanKZLSdX/ctFHiilLYV xz/ehAo3gE0NZgZ8z+S0JK/Sw6oi/UQ8SvYhR8CldYIQdfYNA/0JZuBnaFGUqunjUcClb5 R1WDmi3LXOWcBAUhseI5mdrFVTk4SF/i2wzyxcWKh1MVbUrvJiJzMQnTsjudTOLHo9w2ut SbPgqNSJLc4zn9vQ73czyj1O3ut/A6Q1shBZX1xVrGSYm6MhjUBNqOJxNwNpwQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WQrq40JM2zyDN; Sat, 20 Jul 2024 02:59:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46K2xB8D024654; Sat, 20 Jul 2024 02:59:11 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46K2xBR4024651; Sat, 20 Jul 2024 02:59:11 GMT (envelope-from git) Date: Sat, 20 Jul 2024 02:59:11 GMT Message-Id: <202407200259.46K2xBR4024651@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Warner Losh Subject: git: 9229b3105d88 - main - nvme: Fail passthrough commands right away in failed state. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 9229b3105d88d44012f2c7f85344da42e857ba91 Auto-Submitted: auto-generated The branch main has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=9229b3105d88d44012f2c7f85344da42e857ba91 commit 9229b3105d88d44012f2c7f85344da42e857ba91 Author: Warner Losh AuthorDate: 2024-07-20 02:55:52 +0000 Commit: Warner Losh CommitDate: 2024-07-20 02:55:52 +0000 nvme: Fail passthrough commands right away in failed state. When the drive is failed, we can't send passthrough commands to the card, so fail them right away. Rearrange the comments to reflect the current failure paths in the driver. Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D46048 --- sys/dev/nvme/nvme_qpair.c | 58 ++++++++++++++++++++--------------------------- 1 file changed, 24 insertions(+), 34 deletions(-) diff --git a/sys/dev/nvme/nvme_qpair.c b/sys/dev/nvme/nvme_qpair.c index 5be6c151bbea..9b3fe58c312c 100644 --- a/sys/dev/nvme/nvme_qpair.c +++ b/sys/dev/nvme/nvme_qpair.c @@ -1267,41 +1267,31 @@ _nvme_qpair_submit_request(struct nvme_qpair *qpair, struct nvme_request *req) tr = TAILQ_FIRST(&qpair->free_tr); req->qpair = qpair; - if (tr == NULL || qpair->recovery_state != RECOVERY_NONE) { - /* - * No tracker is available, or the qpair is disabled due to an - * in-progress controller-level reset. If we lose the race with - * recovery_state, then we may add an extra request to the queue - * which will be resubmitted later. We only set recovery_state - * to NONE with qpair->lock also held, so if we observe that the - * state is not NONE, we know it can't transition to NONE below - * when we've submitted the request to hardware. - * - * Also, as part of the failure process, we set recovery_state - * to RECOVERY_WAITING, so we check here to see if we've failed - * the controller. We set it before we call the qpair_fail - * functions, which take out the lock lock before messing with - * queued_req. Since we hold that lock, we know it's safe to - * either fail directly, or queue the failure should is_failed - * be stale. If we lose the race reading is_failed, then - * nvme_qpair_fail will fail the queued request. - */ + /* + * The controller has failed, so fail the request. Note, that this races + * the recovery / timeout code. Since we hold the qpair lock, we know + * it's safe to fail directly. is_failed is set when we fail the controller. + * It is only ever reset in the ioctl reset controller path, which is safe + * to race (for failed controllers, we make no guarantees about bringing + * it out of failed state relative to other commands). + */ + if (qpair->ctrlr->is_failed) { + nvme_qpair_manual_complete_request(qpair, req, + NVME_SCT_GENERIC, NVME_SC_ABORTED_BY_REQUEST); + return; + } - if (qpair->ctrlr->is_failed) { - /* - * The controller has failed, so fail the request. - */ - nvme_qpair_manual_complete_request(qpair, req, - NVME_SCT_GENERIC, NVME_SC_ABORTED_BY_REQUEST); - } else { - /* - * Put the request on the qpair's request queue to be - * processed when a tracker frees up via a command - * completion or when the controller reset is - * completed. - */ - STAILQ_INSERT_TAIL(&qpair->queued_req, req, stailq); - } + /* + * No tracker is available, or the qpair is disabled due to an + * in-progress controller-level reset. If we lose the race with + * recovery_state, then we may add an extra request to the queue which + * will be resubmitted later. We only set recovery_state to NONE with + * qpair->lock also held, so if we observe that the state is not NONE, + * we know it won't transition back to NONE without retrying queued + * request. + */ + if (tr == NULL || qpair->recovery_state != RECOVERY_NONE) { + STAILQ_INSERT_TAIL(&qpair->queued_req, req, stailq); return; }