From nobody Fri Jul 19 16:18:21 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WQZbd2l9zz5RvGB; Fri, 19 Jul 2024 16:18:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WQZbd1mkKz543h; Fri, 19 Jul 2024 16:18:21 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721405901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Dk8krFrepggL7ynLKEvNpR4vfu4ETt66pY1d7tSKppw=; b=ULvV0CP/W94+B/+92DjdBcyDzP0l1ZqiH31E83u07j20VG2HlMp9ltZ9BW2eY5HcWvDZLB /iZfsbGmmhwTCus9LLNqfytze1Jl/WeAiXFjbsQtraht5drrWUQ8zLOSN4+cUg96bssJNS ghCCYhihUHefVt0lCKoA8XBtZ3NIFZ2GU4+19gpFXTRdob0wvKT0Eg+XUfKU+1+r5HbUZL WYlfASHcy/fNA9jYs0n0k7VA+L/y4+KX68hgns/8BbhGwd6TVaso5hQHCgXrRX6dtIHVtc 6+9ce3vFPv2XQktegX4JeXnwOMXMLwrQZdaT/3I1zGxsiHYCxsfn4fBjnJ671g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721405901; a=rsa-sha256; cv=none; b=qQnFmN4kkPh2lPtCcfW2YucGc9WEjuQs7eSdMIcXnmpHwb9snj9fu5oDqHtFvNmfbB8+81 FVK/l07L84oTmWR5dCB7JeF7V92V6NR0L3Tpt6tsmYsMmDNZA7BdVutYvvq02PBfEzeM1s QBfRrUmJmp3Fe33FYEnWkQx9L+e2JuMhDjrxsPG4FELum9cg354yvuDz13d9eFuYr1G67T dwpahk1Gyv8suq0lv9Ww4GyDu8DddONBNod2+rR/1LxuH0D0YjgUcx1e8KZe43wYG7IETn /iK8syAwu5k4oIW674mmWjvZ2jmleQJV4l6TIB+T83vu9PJfL/457vvidixKAQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721405901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Dk8krFrepggL7ynLKEvNpR4vfu4ETt66pY1d7tSKppw=; b=a332s6gZZoAelpjazlnwrkjsDtXFhBUaYmrvwD/FPWYziD+g2Y2T4iWGWFWctJ1kR2GMdR rcd6buPGGxu+NCbIoFhXG3VCqKTRjnkJOkOo/U20VW7YLXPmVbnCLbBysjeiXEdhHz4pDt SwGWFPJ2MzG5vF7Y1UXraCsH4wi2r5KSzjcXP+1Vf/eULDUh1yxAN5EyFbKcRYuHx/H/Rd 6MpAsv4MvJRW9Y8kh6pYowK1SKFoVwDZyGy3+hbuHEGgT38cE5MShC0IXJ0NdpadA5bZvj W+KelJW6d688RKfetJ/Oe2qOt3t/lC4PDKPYBXijWGUl7+h0KYy6tkDt0jCUQQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WQZbd1P2xzfsx; Fri, 19 Jul 2024 16:18:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46JGILwN043460; Fri, 19 Jul 2024 16:18:21 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46JGILP3043457; Fri, 19 Jul 2024 16:18:21 GMT (envelope-from git) Date: Fri, 19 Jul 2024 16:18:21 GMT Message-Id: <202407191618.46JGILP3043457@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Navdeep Parhar Subject: git: 27479403a764 - main - cxgbe/t4_tom: Free up hardware resources when the final CPL is received. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 27479403a764cf3b97194887a1f819c1e35357aa Auto-Submitted: auto-generated The branch main has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=27479403a764cf3b97194887a1f819c1e35357aa commit 27479403a764cf3b97194887a1f819c1e35357aa Author: Navdeep Parhar AuthorDate: 2024-07-16 00:39:40 +0000 Commit: Navdeep Parhar CommitDate: 2024-07-19 16:12:36 +0000 cxgbe/t4_tom: Free up hardware resources when the final CPL is received. Final CPL means the tid is done in the hardware and other resources associated with it can be freed right away. There is no need to wait for the kernel to detach the toepcb. Reviewed by: jhb MFC after: 1 week Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D45991 --- sys/dev/cxgbe/tom/t4_tom.c | 58 +++++++++++++++++++++++++++++++--------------- 1 file changed, 39 insertions(+), 19 deletions(-) diff --git a/sys/dev/cxgbe/tom/t4_tom.c b/sys/dev/cxgbe/tom/t4_tom.c index 10ee78681365..645822b6f781 100644 --- a/sys/dev/cxgbe/tom/t4_tom.c +++ b/sys/dev/cxgbe/tom/t4_tom.c @@ -97,6 +97,7 @@ static struct uld_info tom_uld_info = { }; static void release_offload_resources(struct toepcb *); +static void done_with_toepcb(struct toepcb *); static int alloc_tid_tabs(struct tid_info *); static void free_tid_tabs(struct tid_info *); static void free_tom_data(struct adapter *, struct tom_data *); @@ -311,12 +312,42 @@ release_offload_resources(struct toepcb *toep) KASSERT(!(toep->flags & TPF_CPL_PENDING), ("%s: %p has CPL pending.", __func__, toep)); - KASSERT(!(toep->flags & TPF_ATTACHED), - ("%s: %p is still attached.", __func__, toep)); CTR5(KTR_CXGBE, "%s: toep %p (tid %d, l2te %p, ce %p)", __func__, toep, tid, toep->l2te, toep->ce); + if (toep->l2te) { + t4_l2t_release(toep->l2te); + toep->l2te = NULL; + } + if (tid >= 0) { + remove_tid(sc, tid, toep->ce ? 2 : 1); + release_tid(sc, tid, toep->ctrlq); + toep->tid = -1; + } + if (toep->ce) { + t4_release_clip_entry(sc, toep->ce); + toep->ce = NULL; + } + if (toep->params.tc_idx != -1) + t4_release_cl_rl(sc, toep->vi->pi->port_id, toep->params.tc_idx); +} + +/* + * Both the driver and kernel are done with the toepcb. + */ +static void +done_with_toepcb(struct toepcb *toep) +{ + struct tom_data *td = toep->td; + + KASSERT(!(toep->flags & TPF_CPL_PENDING), + ("%s: %p has CPL pending.", __func__, toep)); + KASSERT(!(toep->flags & TPF_ATTACHED), + ("%s: %p is still attached.", __func__, toep)); + + CTR(KTR_CXGBE, "%s: toep %p (0x%x)", __func__, toep, toep->flags); + /* * These queues should have been emptied at approximately the same time * that a normal connection's socket's so_snd would have been purged or @@ -329,20 +360,9 @@ release_offload_resources(struct toepcb *toep) ddp_assert_empty(toep); #endif MPASS(TAILQ_EMPTY(&toep->aiotx_jobq)); - - if (toep->l2te) - t4_l2t_release(toep->l2te); - - if (tid >= 0) { - remove_tid(sc, tid, toep->ce ? 2 : 1); - release_tid(sc, tid, toep->ctrlq); - } - - if (toep->ce) - t4_release_clip_entry(sc, toep->ce); - - if (toep->params.tc_idx != -1) - t4_release_cl_rl(sc, toep->vi->pi->port_id, toep->params.tc_idx); + MPASS(toep->tid == -1); + MPASS(toep->l2te == NULL); + MPASS(toep->ce == NULL); mtx_lock(&td->toep_list_lock); TAILQ_REMOVE(&td->toep_list, toep, link); @@ -392,7 +412,7 @@ t4_pcb_detach(struct toedev *tod __unused, struct tcpcb *tp) toep->flags &= ~TPF_ATTACHED; if (!(toep->flags & TPF_CPL_PENDING)) - release_offload_resources(toep); + done_with_toepcb(toep); } /* @@ -988,9 +1008,9 @@ final_cpl_received(struct toepcb *toep) toep->flags &= ~(TPF_CPL_PENDING | TPF_WAITING_FOR_FINAL); mbufq_drain(&toep->ulp_pduq); mbufq_drain(&toep->ulp_pdu_reclaimq); - + release_offload_resources(toep); if (!(toep->flags & TPF_ATTACHED)) - release_offload_resources(toep); + done_with_toepcb(toep); if (!in_pcbrele_wlocked(inp)) INP_WUNLOCK(inp);