From nobody Wed Mar 20 22:34:40 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V0Ngj1wlsz5F6hv; Wed, 20 Mar 2024 22:34:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V0Ngh6jDKz4fBd; Wed, 20 Mar 2024 22:34:40 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710974080; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nzPCQbKzKFEKyGtefLlzSYtyk+0e48gxYEBydI0SgBo=; b=UAflxD0y9XRFBHmL/+QrFPKPBz0r96uZdOhqTXeTl2esSAJ6GIS4pfi6z9koyH77ywoO87 EzD96K8hEdzmlvnRVSlwXg5qXxXUMj5u3GiHIuPCVgCzotIDP1qxN3TYQCDcnM7nfJMdxx vrTLY7E1uwV7Cd/oCcaSqLH/ApStWmhUUMRS8XatFC6l1X30oFQkfQjBdJQPb6YBXy9pvU 1zG4bRftdDyJBGftGs0kgctIyB8MTE8KsNVhHPvZ9evqXdVC3ECDQ4GAuspjg93vRwpovy Bmwctyps036LbJ8gUn0ecoIviJ3FcYheJvNWNuoOVqSvccwgwccKDtIbe9WpPw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710974080; a=rsa-sha256; cv=none; b=lO/OZQUCJj8ZEiXRzMvJH5ZphH4JvBxkvqJz5G0/DmfNdsus2j6PsOge3X27kOdTYZ/tZN MnhRYSniQCl8JfSXjT+llVKeZzxUIdZNKT6C/vWjjbRZoeMrGwIauZjp9zeig3NtF9Jsiu KJqs2bPPQbrevEd0eh5KlyKtztGqLK27fSF2SsuPd9VEsinMsPPXFwIXPoQL/JMUDJPdHF ExfsuVHP2Ab8sJKRRCGfn4d/ZaYsUktOWqZGhhUpCTXazm8l0UP1whi3+M8SYFFEOmfecs pE3eWZvnu5MJASD7QCSJl0vVU2BR0cXQ9Ro/x5TIUQZIH5uSy05Cnsi6qFLmIg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710974080; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nzPCQbKzKFEKyGtefLlzSYtyk+0e48gxYEBydI0SgBo=; b=FmOQdB4SyME6gmuYWYclXemlQpZ7Uv82E1ZQlNa1hlxZClHRqO8xL2QU6R3vHiFmDnUnfU sejaVEHUVfWDA3M718ZA7sbGQljCQPo+HrUDigUTzky4akU+I/7PKwoOt3TVx9/33fE7eP 5g3Z2yuRMoMjNYcVkyb1jVxov71pYSF8yIo1zbbh6e1H/b+X+v1ZghC7MRclT+sjMIKGrA 5cpHOQzFP22OfpFVlQlQoEPoYsxtOW7DX173JuaOvbAb0r382G3zBlq7MJ13DX1Kv/3AhD kiCVeM2OX5/xrGEiwwvMuJU5hmoO1T1WFjXhGj+PIw2ecc90f2JwtLmhu9JgOw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4V0Ngh6DZHz18cx; Wed, 20 Mar 2024 22:34:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 42KMYeQK048013; Wed, 20 Mar 2024 22:34:40 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 42KMYev9048010; Wed, 20 Mar 2024 22:34:40 GMT (envelope-from git) Date: Wed, 20 Mar 2024 22:34:40 GMT Message-Id: <202403202234.42KMYev9048010@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 9978c6289d62 - main - cxgbe tom: Handle a race condition when enabling TLS offload List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 9978c6289d621ac9edc95acb4e0f527a62a49b03 Auto-Submitted: auto-generated The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=9978c6289d621ac9edc95acb4e0f527a62a49b03 commit 9978c6289d621ac9edc95acb4e0f527a62a49b03 Author: John Baldwin AuthorDate: 2024-03-20 22:30:09 +0000 Commit: John Baldwin CommitDate: 2024-03-20 22:30:09 +0000 cxgbe tom: Handle a race condition when enabling TLS offload Use a separate state for when a request to set RX_QUIESCE has been sent but the resulting TCB reply has not been received. In particular, this correctly handles the case where data has been received and queued in the receive queue before the quiesce request takes effect. Reviewed by: np Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D44435 --- sys/dev/cxgbe/tom/t4_tls.c | 12 ++++++++++-- sys/dev/cxgbe/tom/t4_tom.h | 5 +++-- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/sys/dev/cxgbe/tom/t4_tls.c b/sys/dev/cxgbe/tom/t4_tls.c index 3a0c1a392e6c..bdd03edd3a6f 100644 --- a/sys/dev/cxgbe/tom/t4_tls.c +++ b/sys/dev/cxgbe/tom/t4_tls.c @@ -315,7 +315,7 @@ tls_alloc_ktls(struct toepcb *toep, struct ktls_session *tls, int direction) tls->params.max_frame_len; toep->tls.tx_key_info_size = t4_tls_key_info_size(tls); } else { - toep->flags |= TPF_TLS_STARTING | TPF_TLS_RX_QUIESCED; + toep->flags |= TPF_TLS_STARTING | TPF_TLS_RX_QUIESCING; toep->tls.rx_version = tls->params.tls_vmajor << 8 | tls->params.tls_vminor; @@ -1243,6 +1243,10 @@ tls_received_starting_data(struct adapter *sc, struct toepcb *toep, { MPASS(toep->flags & TPF_TLS_STARTING); + /* Data was received before quiescing took effect. */ + if ((toep->flags & TPF_TLS_RX_QUIESCING) != 0) + return; + /* * A previous call to tls_check_rx_sockbuf needed more data. * Now that more data has arrived, quiesce receive again and @@ -1250,7 +1254,7 @@ tls_received_starting_data(struct adapter *sc, struct toepcb *toep, */ if ((toep->flags & TPF_TLS_RX_QUIESCED) == 0) { CTR(KTR_CXGBE, "%s: tid %d quiescing", __func__, toep->tid); - toep->flags |= TPF_TLS_RX_QUIESCED; + toep->flags |= TPF_TLS_RX_QUIESCING; t4_set_rx_quiesce(toep); return; } @@ -1287,6 +1291,10 @@ do_tls_tcb_rpl(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) if ((toep->flags & TPF_TLS_STARTING) == 0) panic("%s: connection is not starting TLS RX\n", __func__); + MPASS((toep->flags & TPF_TLS_RX_QUIESCING) != 0); + + toep->flags &= ~TPF_TLS_RX_QUIESCING; + toep->flags |= TPF_TLS_RX_QUIESCED; so = inp->inp_socket; sb = &so->so_rcv; diff --git a/sys/dev/cxgbe/tom/t4_tom.h b/sys/dev/cxgbe/tom/t4_tom.h index be810d6364b1..805d8bc0d644 100644 --- a/sys/dev/cxgbe/tom/t4_tom.h +++ b/sys/dev/cxgbe/tom/t4_tom.h @@ -73,8 +73,9 @@ enum { TPF_KTLS = (1 << 11), /* send TLS records from KTLS */ TPF_INITIALIZED = (1 << 12), /* init_toepcb has been called */ TPF_TLS_RECEIVE = (1 << 13), /* should receive TLS records */ - TPF_TLS_RX_QUIESCED = (1 << 14), /* RX quiesced for TLS RX startup */ - TPF_WAITING_FOR_FINAL = (1<< 15), /* waiting for wakeup on final CPL */ + TPF_TLS_RX_QUIESCING = (1 << 14), /* RX quiesced for TLS RX startup */ + TPF_TLS_RX_QUIESCED = (1 << 15), /* RX quiesced for TLS RX startup */ + TPF_WAITING_FOR_FINAL = (1<< 16), /* waiting for wakeup on final CPL */ }; enum {