From nobody Mon Jul 22 03:57:28 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WS61N5XpSz5RDgQ; Mon, 22 Jul 2024 03:57:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WS61N3hryz41Cf; Mon, 22 Jul 2024 03:57:28 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721620648; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=d6tJKvqlESv2g54MSleqssM40HF4RJv3ITfAz7XKhCU=; b=GF9SXob1EvtQ+u41feMH0LgYkuHs/UZROJk06MZ/HjwPlco47tsCOiDOnUACyJY5k0Qfax Px1MJ0cSVV3+1VGC7qfTKZaHYjhwpTULLvmQpEUNN1G2SXN4QGnO7ViSiNteRhm0F+LVmk cykZR4W+aVpMQtMKdK0ZdedsryUiKyBT2zs0X0roQOz8Op1P3wxBFnOxEC3Yi3EQmF9dEB Vjg8txVUmTk2krlM4QVEczAjX9RRCXmDN6lBxvi/s9Wl5zUMal2f5wjwT13cFH3Kw+C4XO UbFhU1di6FM9FPDVKZ27/oDEYx21isOtKZXnwYcYFNZlsCAVzsPqo0uYZGsvQg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721620648; a=rsa-sha256; cv=none; b=iASKwmF53Tam+qT3W++ur1QY99WQ1dBaJDxz6Zkl81Deo2y0E1wakoBx1lkIaBB4var8v0 JCbv8D0WQ3DQHv6HF4kVnjLPgrnQZDS3gXA6/n07jxzX/UE+fW41JrVYFUABbODBsqWafh ubdGzLpAKKL1+Djw2EMnGMQzGjOjy1u1aobu/WwwmMud0a5TT8L34UR/9ud5fNfJ31V5eX XXj9g5MWHIY2uQ4sIHs3uLpt4zaVBN40H0fjY8U6vLDEQ7MGEyyNfOcbaPkdcHsoXPcRjC H+vltnb9352mwlw8uFB3Cv/zo5vXVpqlDaOr3OAU+rJ9xX6Z1NDWelCnpNhNSg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721620648; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=d6tJKvqlESv2g54MSleqssM40HF4RJv3ITfAz7XKhCU=; b=aslnnizen0FURn0fevksYBsZurKxzqrJhpmh7zOZgd5Juo/jsEkyuJT+cap13iY/K4ZCDO 5E4zrG7cLCwP6aZqfvUF70osCHEg9VRQ85W2dITkUIGRkfX/HskOr7vuvxf0gG1u55NkJf zl6Jgm0NY7UCqvwhipPVZ0WZ0dsDMci0ag7ScQjQF/oxcAwuIcUzUj74XqGZUAzXcH658x 43EsyCEGeHSPM8nemDYHNCy+yeunPW1MhKeULVzW1fSRv/ApSTtb+C5e8KeU3qxPC/4eBP JrHzA1HQHWyt2gULm6ohKHrZ7PNlCOwsw0791gpBSLUBGI8ddCSQu/y858xxcg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WS61N3JjNzSvl; Mon, 22 Jul 2024 03:57:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46M3vS8E014997; Mon, 22 Jul 2024 03:57:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46M3vS2v014994; Mon, 22 Jul 2024 03:57:28 GMT (envelope-from git) Date: Mon, 22 Jul 2024 03:57:28 GMT Message-Id: <202407220357.46M3vS2v014994@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Navdeep Parhar Subject: git: cd93fdee5c8b - main - cxgbe(4): Stop and restart the L2T allocator with the LLD. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: cd93fdee5c8bbdb00d10f8a1fa43f30f151a1ef7 Auto-Submitted: auto-generated The branch main has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=cd93fdee5c8bbdb00d10f8a1fa43f30f151a1ef7 commit cd93fdee5c8bbdb00d10f8a1fa43f30f151a1ef7 Author: Navdeep Parhar AuthorDate: 2024-07-19 16:40:44 +0000 Commit: Navdeep Parhar CommitDate: 2024-07-22 03:29:26 +0000 cxgbe(4): Stop and restart the L2T allocator with the LLD. L2T entries are used by both filters and TOE and the L2T is shared between the base driver (LLD) and the TOM ULD. Add a flag to indicate that the L2T is stopped, which means: * t4_alloc_l2e and t4_l2t_alloc_switching will not allocate new entries. * t4_tom will ignore all ARP/NDP updates from the kernel. * Previously allocated L2T entries can still be freed. MFC after: 1 month Sponsored by: Chelsio Communications --- sys/dev/cxgbe/t4_l2t.c | 36 +++++++++++++++++++++++++++++++++--- sys/dev/cxgbe/t4_l2t.h | 5 ++++- sys/dev/cxgbe/t4_main.c | 4 +++- sys/dev/cxgbe/tom/t4_tom_l2t.c | 7 +++++++ 4 files changed, 47 insertions(+), 5 deletions(-) diff --git a/sys/dev/cxgbe/t4_l2t.c b/sys/dev/cxgbe/t4_l2t.c index 90f184f5e666..b210003cfac0 100644 --- a/sys/dev/cxgbe/t4_l2t.c +++ b/sys/dev/cxgbe/t4_l2t.c @@ -73,7 +73,8 @@ t4_alloc_l2e(struct l2t_data *d) struct l2t_entry *end, *e, **p; rw_assert(&d->lock, RA_WLOCKED); - + if (__predict_false(d->l2t_stopped)) + return (NULL); if (!atomic_load_acq_int(&d->nfree)) return (NULL); @@ -291,7 +292,10 @@ t4_l2t_alloc_switching(struct adapter *sc, uint16_t vlan, uint8_t port, int rc; rw_wlock(&d->lock); - e = find_or_alloc_l2e(d, vlan, port, eth_addr); + if (__predict_false(d->l2t_stopped)) + e = NULL; + else + e = find_or_alloc_l2e(d, vlan, port, eth_addr); if (e) { if (atomic_load_acq_int(&e->refcnt) == 0) { mtx_lock(&e->lock); /* avoid race with t4_l2t_free */ @@ -333,6 +337,7 @@ t4_init_l2t(struct adapter *sc, int flags) return (ENOMEM); d->l2t_size = l2t_size; + d->l2t_stopped = false; d->rover = d->l2tab; atomic_store_rel_int(&d->nfree, l2t_size); rw_init(&d->lock, "L2T"); @@ -353,8 +358,9 @@ t4_init_l2t(struct adapter *sc, int flags) } int -t4_free_l2t(struct l2t_data *d) +t4_free_l2t(struct adapter *sc) { + struct l2t_data *d = sc->l2t; int i; for (i = 0; i < d->l2t_size; i++) @@ -365,6 +371,30 @@ t4_free_l2t(struct l2t_data *d) return (0); } +int +t4_stop_l2t(struct adapter *sc) +{ + struct l2t_data *d = sc->l2t; + + rw_wlock(&d->lock); + d->l2t_stopped = true; + rw_wunlock(&d->lock); + + return (0); +} + +int +t4_restart_l2t(struct adapter *sc) +{ + struct l2t_data *d = sc->l2t; + + rw_wlock(&d->lock); + d->l2t_stopped = false; + rw_wunlock(&d->lock); + + return (0); +} + int do_l2t_write_rpl(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) diff --git a/sys/dev/cxgbe/t4_l2t.h b/sys/dev/cxgbe/t4_l2t.h index 991b4476eb61..f8681a4ab556 100644 --- a/sys/dev/cxgbe/t4_l2t.h +++ b/sys/dev/cxgbe/t4_l2t.h @@ -79,6 +79,7 @@ struct l2t_entry { struct l2t_data { struct rwlock lock; u_int l2t_size; + bool l2t_stopped; volatile int nfree; /* number of free entries */ struct l2t_entry *rover;/* starting point for next allocation */ struct l2t_entry l2tab[]; @@ -86,7 +87,9 @@ struct l2t_data { int t4_init_l2t(struct adapter *, int); -int t4_free_l2t(struct l2t_data *); +int t4_free_l2t(struct adapter *); +int t4_stop_l2t(struct adapter *); +int t4_restart_l2t(struct adapter *); struct l2t_entry *t4_alloc_l2e(struct l2t_data *); struct l2t_entry *t4_l2t_alloc_switching(struct adapter *, uint16_t, uint8_t, uint8_t *); diff --git a/sys/dev/cxgbe/t4_main.c b/sys/dev/cxgbe/t4_main.c index f1f4b2d26fdf..3baa2d5ac0f7 100644 --- a/sys/dev/cxgbe/t4_main.c +++ b/sys/dev/cxgbe/t4_main.c @@ -1847,7 +1847,7 @@ t4_detach_common(device_t dev) sc->msix_res); if (sc->l2t) - t4_free_l2t(sc->l2t); + t4_free_l2t(sc); if (sc->smt) t4_free_smt(sc->smt); t4_free_atid_table(sc); @@ -2101,6 +2101,7 @@ stop_lld(struct adapter *sc) end_synchronized_op(sc, 0); stop_atid_allocator(sc); + t4_stop_l2t(sc); return (rc); } @@ -2454,6 +2455,7 @@ done: free(old_state, M_CXGBE); restart_atid_allocator(sc); + t4_restart_l2t(sc); return (rc); } diff --git a/sys/dev/cxgbe/tom/t4_tom_l2t.c b/sys/dev/cxgbe/tom/t4_tom_l2t.c index ee442f023970..749e5704863e 100644 --- a/sys/dev/cxgbe/tom/t4_tom_l2t.c +++ b/sys/dev/cxgbe/tom/t4_tom_l2t.c @@ -380,6 +380,10 @@ t4_l2t_get(struct port_info *pi, if_t ifp, struct sockaddr *sa) hash = l2_hash(d, sa, if_getindex(ifp)); rw_wlock(&d->lock); + if (__predict_false(d->l2t_stopped)) { + e = NULL; + goto done; + } for (e = d->l2tab[hash].first; e; e = e->next) { if (l2_cmp(sa, e) == 0 && e->ifp == ifp && e->vlan == vtag && e->smt_idx == smt_idx) { @@ -429,6 +433,8 @@ t4_l2_update(struct toedev *tod, if_t ifp, struct sockaddr *sa, hash = l2_hash(d, sa, if_getindex(ifp)); rw_rlock(&d->lock); + if (__predict_false(d->l2t_stopped)) + goto done; for (e = d->l2tab[hash].first; e; e = e->next) { if (l2_cmp(sa, e) == 0 && e->ifp == ifp) { mtx_lock(&e->lock); @@ -439,6 +445,7 @@ t4_l2_update(struct toedev *tod, if_t ifp, struct sockaddr *sa, break; } } +done: rw_runlock(&d->lock); /*