From nobody Mon Oct 21 18:39:18 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XXPGt5Jnyz5ZcYS; Mon, 21 Oct 2024 18:39:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XXPGt454Nz4G49; Mon, 21 Oct 2024 18:39:18 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1729535958; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fyJIGzfalcs1CJJj8ULUQKN9Hg9sCIlGos8XJWKL92o=; b=bG9o04NT7qjhaqvOgJrdH6i0g8kUDuFTE8kKdNeWf3sR26etvVGKVY3UZF6bB9ZFUmGeRX m7W1zL+BVsYGptdOUG3R03lRXGnHQHuK/66lulI4PMcw1ZfaKbVZ1G4oUfRjtovslEzRO1 sTHGVOoDWA7EnlURyX39ncl2rBDLUFjItCmrp7+C3ldg6q2X6T8pOvHYvAdv9zlT43VKiG OrimkrGNF7u3rT7Eeapbm8gmmQ0eUEjR8LeZa2PKJsM69iZIH+vEzMj3LKeu76nxpurzJ8 p3okw98INYMopu+I673lR17C4uDpcXOo56GCHtD53UHL7jUDXYOMR8DgolhEVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1729535958; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fyJIGzfalcs1CJJj8ULUQKN9Hg9sCIlGos8XJWKL92o=; b=o4akQVjcB9fcqpBT5/84sktVW78Rjz3NVjWspn+n0Z6TEzlWrVO6FJqqUpOfJ4gWMb/YQY 1K0PxqfKrbcQoAZPIdRQDWIzCHPfKseN2b4ZIMFru9tKMiOsI9aXCm8iVMAYqbg3dhZlMe ee03JcwlaLqocWIInlehBCAmANv1NlO0kiTV/fDGUjMDwm+NjWRVkd71HX+IKOqIr2Q0JU W1hIbzHqprO2UMm4KOzSWDKdS/bSDRvTwLZ6o4nZ4go1DX9TfN+0sIZHGqmPWrue8SHJb9 yRGo0RV7wnc7SZvsb5gJp4aq9JNjZI4iF554is0O9/3aMKxsgbx4EAsCdpIK+g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1729535958; a=rsa-sha256; cv=none; b=TLAMfgqdEBYkv9yUUlXXzSY8WCkj+C0b2GfbZfZAnDDdrKyyewaiB2oT4WlmiOE8H700TK tTPG9Z1kIPkdWKQnQAd0hNEGBmeHsPLYkJFK3Fu3aEGJ8py4AebSsHpnM2wBsFsHcCLLk/ x/LNOYJajKVv6+4Yy2pfFuqFhU6MaGZ9bfTspy1fX3Ef3WADb04EVdpkD0el+ZxuvftZcp bLgd0hkddWDyc5rjrgv02+Pb66jsNUhoDNTO9VZxc9QALrSGAdKYYw1saBGq9XkSy7nfNE ih9S4zkkRDM+V91h9XbVsMgJA0MW+I5F7jq3vfzRjvpOmubMKl+6uFq14Mef0A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XXPGt2Gg8zh88; Mon, 21 Oct 2024 18:39:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 49LIdI0t036503; Mon, 21 Oct 2024 18:39:18 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 49LIdIEe036500; Mon, 21 Oct 2024 18:39:18 GMT (envelope-from git) Date: Mon, 21 Oct 2024 18:39:18 GMT Message-Id: <202410211839.49LIdIEe036500@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Navdeep Parhar Subject: git: 888858a7bfbf - stable/14 - cxgbe(4): Stop work request queues in a reliable manner. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 888858a7bfbf3112ea314cc0d7f12d385c4c39de Auto-Submitted: auto-generated The branch stable/14 has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=888858a7bfbf3112ea314cc0d7f12d385c4c39de commit 888858a7bfbf3112ea314cc0d7f12d385c4c39de Author: Navdeep Parhar AuthorDate: 2024-07-31 19:27:18 +0000 Commit: Navdeep Parhar CommitDate: 2024-10-21 17:06:19 +0000 cxgbe(4): Stop work request queues in a reliable manner. Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all work requests, pending or new, when it's not set. Sponsored by: Chelsio Communications (cherry picked from commit 0a9d1da6e6cede5e9c0ff63240d724049ad72b5b) --- sys/dev/cxgbe/adapter.h | 5 ++++- sys/dev/cxgbe/t4_main.c | 20 +++++++++++++++++++- sys/dev/cxgbe/t4_sge.c | 15 ++++++++++++++- 3 files changed, 37 insertions(+), 3 deletions(-) diff --git a/sys/dev/cxgbe/adapter.h b/sys/dev/cxgbe/adapter.h index 0d731e736823..3922bd3909fe 100644 --- a/sys/dev/cxgbe/adapter.h +++ b/sys/dev/cxgbe/adapter.h @@ -1561,7 +1561,10 @@ t4_wrq_tx(struct adapter *sc, struct wrqe *wr) struct sge_wrq *wrq = wr->wrq; TXQ_LOCK(wrq); - t4_wrq_tx_locked(sc, wrq, wr); + if (__predict_true(wrq->eq.flags & EQ_HW_ALLOCATED)) + t4_wrq_tx_locked(sc, wrq, wr); + else + free(wr, M_CXGBE); TXQ_UNLOCK(wrq); } diff --git a/sys/dev/cxgbe/t4_main.c b/sys/dev/cxgbe/t4_main.c index c02fe8372946..4a9e64efcacf 100644 --- a/sys/dev/cxgbe/t4_main.c +++ b/sys/dev/cxgbe/t4_main.c @@ -2060,7 +2060,9 @@ stop_lld(struct adapter *sc) } #if defined(TCP_OFFLOAD) || defined(RATELIMIT) for_each_ofld_txq(vi, k, ofld_txq) { + TXQ_LOCK(&ofld_txq->wrq); ofld_txq->wrq.eq.flags &= ~EQ_HW_ALLOCATED; + TXQ_UNLOCK(&ofld_txq->wrq); } #endif for_each_rxq(vi, k, rxq) { @@ -2078,7 +2080,9 @@ stop_lld(struct adapter *sc) if (sc->flags & FULL_INIT_DONE) { /* Control queue */ wrq = &sc->sge.ctrlq[i]; + TXQ_LOCK(wrq); wrq->eq.flags &= ~EQ_HW_ALLOCATED; + TXQ_UNLOCK(wrq); quiesce_wrq(wrq); } } @@ -7049,8 +7053,22 @@ quiesce_txq(struct sge_txq *txq) static void quiesce_wrq(struct sge_wrq *wrq) { + struct wrqe *wr; - /* XXXTX */ + TXQ_LOCK(wrq); + while ((wr = STAILQ_FIRST(&wrq->wr_list)) != NULL) { + STAILQ_REMOVE_HEAD(&wrq->wr_list, link); +#ifdef INVARIANTS + wrq->nwr_pending--; + wrq->ndesc_needed -= howmany(wr->wr_len, EQ_ESIZE); +#endif + free(wr, M_CXGBE); + } + MPASS(wrq->nwr_pending == 0); + MPASS(wrq->ndesc_needed == 0); + wrq->nwr_pending = 0; + wrq->ndesc_needed = 0; + TXQ_UNLOCK(wrq); } static void diff --git a/sys/dev/cxgbe/t4_sge.c b/sys/dev/cxgbe/t4_sge.c index 614aa1f7508c..cc927f27d318 100644 --- a/sys/dev/cxgbe/t4_sge.c +++ b/sys/dev/cxgbe/t4_sge.c @@ -2921,6 +2921,10 @@ start_wrq_wr(struct sge_wrq *wrq, int len16, struct wrq_cookie *cookie) MPASS(ndesc > 0 && ndesc <= SGE_MAX_WR_NDESC); EQ_LOCK(eq); + if (__predict_false((eq->flags & EQ_HW_ALLOCATED) == 0)) { + EQ_UNLOCK(eq); + return (NULL); + } if (TAILQ_EMPTY(&wrq->incomplete_wrs) && !STAILQ_EMPTY(&wrq->wr_list)) drain_wrq_wr_list(sc, wrq); @@ -3016,7 +3020,10 @@ commit_wrq_wr(struct sge_wrq *wrq, void *w, struct wrq_cookie *cookie) F_FW_WR_EQUEQ); } - ring_eq_db(wrq->adapter, eq, ndesc); + if (__predict_true(eq->flags & EQ_HW_ALLOCATED)) + ring_eq_db(wrq->adapter, eq, ndesc); + else + IDXINCR(eq->dbidx, ndesc, eq->sidx); } else { MPASS(IDXDIFF(next->pidx, pidx, eq->sidx) == ndesc); next->pidx = pidx; @@ -3852,6 +3859,8 @@ alloc_ctrlq(struct adapter *sc, int idx) if (!(ctrlq->eq.flags & EQ_HW_ALLOCATED)) { MPASS(ctrlq->eq.flags & EQ_SW_ALLOCATED); + MPASS(ctrlq->nwr_pending == 0); + MPASS(ctrlq->ndesc_needed == 0); rc = alloc_eq_hwq(sc, NULL, &ctrlq->eq); if (rc != 0) { @@ -4556,6 +4565,7 @@ free_wrq(struct adapter *sc, struct sge_wrq *wrq) { free_eq(sc, &wrq->eq); MPASS(wrq->nwr_pending == 0); + MPASS(wrq->ndesc_needed == 0); MPASS(TAILQ_EMPTY(&wrq->incomplete_wrs)); MPASS(STAILQ_EMPTY(&wrq->wr_list)); bzero(wrq, sizeof(*wrq)); @@ -4850,6 +4860,9 @@ alloc_ofld_txq(struct vi_info *vi, struct sge_ofld_txq *ofld_txq, int idx) } if (!(eq->flags & EQ_HW_ALLOCATED)) { + MPASS(eq->flags & EQ_SW_ALLOCATED); + MPASS(ofld_txq->wrq.nwr_pending == 0); + MPASS(ofld_txq->wrq.ndesc_needed == 0); rc = alloc_eq_hwq(sc, vi, eq); if (rc != 0) { CH_ERR(vi, "failed to create hw ofld_txq%d: %d\n", idx,