From nobody Mon Apr 08 19:04:48 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VCz6m3pJRz5HYqN; Mon, 8 Apr 2024 19:04:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VCz6m17Htz4h6F; Mon, 8 Apr 2024 19:04:48 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1712603088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=7nG3wlrWthJ7ckAhASJxf1S1MRvuFI0wADbYEDE1018=; b=MksgaKm+4s1evvuQX1VMdL3d3BvgGGB+hlFyFSLa4uQmFEXL1rxovnRVcDTFIkY0T5ypHx toB9AJzbZsU0MdZRJ08VHB0p63Rw+zmNgoJdDacqzP0HP1H4f2I/M/lUj5jNibrq5q3Dny EVU2T9dxXxCk9GhlQfE5RKUXxyOp8jMFDjlqfTAeM7CkMkFdgfr3bY2TcT3Rr482hFUcoD rM6LAjjwFGsmc17dbM303wFbUtvg/X+F4TmdBEm4R7i3r5eFgnKAHDo3JXyhdKWE8dVcEf +G2T/qaqpoqnfGbgw2luXr7U+OH+LPPku5VKi4gvPZvlqdZMQ6TU/s5He4DVOg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1712603088; a=rsa-sha256; cv=none; b=pughCxDfhyFwvPzeHsUDy622+z6x92dXCGSpl1O9lmTdeY5D/MBAWGMJ5M7BzkFaA9Uajl DVTNLxt0xlRjVLAo1bY3p4Ao6+2jAaYyeOxpOqpIJ5lV/Fb1HiUWBtUUKf3Qo2evm066GG Pv+OjclL33AKYw8tVovla++Fe38FXREKiPjniF4VR0d0hlyrzUTM9+GBUy2n2Mwu+TMVX5 zfhZhy06Nfs9zoYg4mRA4bBEMAHaPWpOm5K35uOZ5nAr4XRC63D5JZVhEwPF6QrKdLTMGr KySRzkzOv8HfFLFk8Td61Zz+igE8VO6DfiygYE+gh/uFfcRigPnGPTdaCWqwgw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1712603088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=7nG3wlrWthJ7ckAhASJxf1S1MRvuFI0wADbYEDE1018=; b=wyyEMIjU7XFxdTdTG+KdWE2HnZeqFKxNc4icBqqCE7mxZ2uJIPWjXqWsM3EN+hBGEyEAqW bSWGJTtrbYXODeQlmYns8luGiSLi1CIlrZZYOimbXd9GkgCxEAzDMU+/zsfWYFT5OTNgZK VcSdBdGalvb0AHC56dtAaMqEVaNoLbGQJtmAT869SI5IvTn6u64Xy4Re7Fb5wbUtC+htFm zZQGxfzvYAy/I79kdBUDrzReMulE/4wocMQ13q3vCLRzPGot/pb5LzzdKuRNIeY7WbB0WT qt7wdao1sDX3BXxjaz1GwkqwmQJnlTedsmZ1HWcoArCuExub2bu6nAB/wMvz4Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VCz6m0lZXzNSp; Mon, 8 Apr 2024 19:04:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 438J4m9p095179; Mon, 8 Apr 2024 19:04:48 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 438J4mth095176; Mon, 8 Apr 2024 19:04:48 GMT (envelope-from git) Date: Mon, 8 Apr 2024 19:04:48 GMT Message-Id: <202404081904.438J4mth095176@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: 7748f70561f3 - stable/14 - cxgbe tom: Handle a race condition when enabling TLS offload List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 7748f70561f373a8d8e70c32223c99b02904ef51 Auto-Submitted: auto-generated The branch stable/14 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=7748f70561f373a8d8e70c32223c99b02904ef51 commit 7748f70561f373a8d8e70c32223c99b02904ef51 Author: John Baldwin AuthorDate: 2024-03-20 22:30:09 +0000 Commit: John Baldwin CommitDate: 2024-04-08 18:07:13 +0000 cxgbe tom: Handle a race condition when enabling TLS offload Use a separate state for when a request to set RX_QUIESCE has been sent but the resulting TCB reply has not been received. In particular, this correctly handles the case where data has been received and queued in the receive queue before the quiesce request takes effect. Reviewed by: np Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D44435 (cherry picked from commit 9978c6289d621ac9edc95acb4e0f527a62a49b03) --- sys/dev/cxgbe/tom/t4_tls.c | 12 ++++++++++-- sys/dev/cxgbe/tom/t4_tom.h | 5 +++-- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/sys/dev/cxgbe/tom/t4_tls.c b/sys/dev/cxgbe/tom/t4_tls.c index 3a0c1a392e6c..bdd03edd3a6f 100644 --- a/sys/dev/cxgbe/tom/t4_tls.c +++ b/sys/dev/cxgbe/tom/t4_tls.c @@ -315,7 +315,7 @@ tls_alloc_ktls(struct toepcb *toep, struct ktls_session *tls, int direction) tls->params.max_frame_len; toep->tls.tx_key_info_size = t4_tls_key_info_size(tls); } else { - toep->flags |= TPF_TLS_STARTING | TPF_TLS_RX_QUIESCED; + toep->flags |= TPF_TLS_STARTING | TPF_TLS_RX_QUIESCING; toep->tls.rx_version = tls->params.tls_vmajor << 8 | tls->params.tls_vminor; @@ -1243,6 +1243,10 @@ tls_received_starting_data(struct adapter *sc, struct toepcb *toep, { MPASS(toep->flags & TPF_TLS_STARTING); + /* Data was received before quiescing took effect. */ + if ((toep->flags & TPF_TLS_RX_QUIESCING) != 0) + return; + /* * A previous call to tls_check_rx_sockbuf needed more data. * Now that more data has arrived, quiesce receive again and @@ -1250,7 +1254,7 @@ tls_received_starting_data(struct adapter *sc, struct toepcb *toep, */ if ((toep->flags & TPF_TLS_RX_QUIESCED) == 0) { CTR(KTR_CXGBE, "%s: tid %d quiescing", __func__, toep->tid); - toep->flags |= TPF_TLS_RX_QUIESCED; + toep->flags |= TPF_TLS_RX_QUIESCING; t4_set_rx_quiesce(toep); return; } @@ -1287,6 +1291,10 @@ do_tls_tcb_rpl(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) if ((toep->flags & TPF_TLS_STARTING) == 0) panic("%s: connection is not starting TLS RX\n", __func__); + MPASS((toep->flags & TPF_TLS_RX_QUIESCING) != 0); + + toep->flags &= ~TPF_TLS_RX_QUIESCING; + toep->flags |= TPF_TLS_RX_QUIESCED; so = inp->inp_socket; sb = &so->so_rcv; diff --git a/sys/dev/cxgbe/tom/t4_tom.h b/sys/dev/cxgbe/tom/t4_tom.h index d74f3f908286..ddf7dec55784 100644 --- a/sys/dev/cxgbe/tom/t4_tom.h +++ b/sys/dev/cxgbe/tom/t4_tom.h @@ -73,8 +73,9 @@ enum { TPF_KTLS = (1 << 11), /* send TLS records from KTLS */ TPF_INITIALIZED = (1 << 12), /* init_toepcb has been called */ TPF_TLS_RECEIVE = (1 << 13), /* should receive TLS records */ - TPF_TLS_RX_QUIESCED = (1 << 14), /* RX quiesced for TLS RX startup */ - TPF_WAITING_FOR_FINAL = (1<< 15), /* waiting for wakeup on final CPL */ + TPF_TLS_RX_QUIESCING = (1 << 14), /* RX quiesced for TLS RX startup */ + TPF_TLS_RX_QUIESCED = (1 << 15), /* RX quiesced for TLS RX startup */ + TPF_WAITING_FOR_FINAL = (1<< 16), /* waiting for wakeup on final CPL */ }; enum {