From nobody Mon Jan 24 22:20:23 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 85ABD197D3C9; Mon, 24 Jan 2022 22:20:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JjPYz2rTsz3lHt; Mon, 24 Jan 2022 22:20:23 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643062823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=raXAJIt4HxvJcgmqsBodIa48mXtKXjyP5228Fx8CyQc=; b=mEkL9Rs+RMUE2y81aNzdR6CZiXk+PEYGu0pURdYgEfC63MdhMIyHHO0x/MiMPAfLg9v4n0 ixC4UZmrowJasZHzWIYQqJ6sX51TH9aRQb4jYN1Kc7iZMOS5tYEb4XdIq8v8UmAO0kw3iG rL1OPXqeUo+10/vnwmrYbEg4KXsdn8rgMZXZReGvjsMPAfljFwsx3M92aJFylGOk5s1SIz h8bdnOvP0ypRhZx2epno++2dPgUM2Ewzln4vtlHiHjJ41sGbbaOBSHfZSXREjn8ZDg0hut 4PDbo5LDhhQBgL3RGn2VieZki5rrS1b36UDGLFizQtOXipljzc11cVbt5b057A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 319141E104; Mon, 24 Jan 2022 22:20:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20OMKN69084276; Mon, 24 Jan 2022 22:20:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20OMKNIk084275; Mon, 24 Jan 2022 22:20:23 GMT (envelope-from git) Date: Mon, 24 Jan 2022 22:20:23 GMT Message-Id: <202201242220.20OMKNIk084275@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: a8c4147edcdc - main - cxgbei: Parse all PDUs received prior to enabling offload mode. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a8c4147edcdce934f93dd848c6ed083500dff22c Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643062823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=raXAJIt4HxvJcgmqsBodIa48mXtKXjyP5228Fx8CyQc=; b=pAaspJqoMtyEz4gFtuJ+TufJFOabukhoyafBLZt5qMWEJLoS1EVdL2YJ6033Y7/V0p2wSH LzIkab+8Ew2wCTFJYo/QCZ+eWFrFU9b6xMR6AAtvpZ+SIAVwAKhBjiaH1t5LQpUgCKlsuv NN2H3Ok6iUIsPPhTehDbOw5nFRfeuj1SuGBIgkLutoNobGihs1eSRGsuT23ZLTTFU4Hr5Q JUY+jTPbxwd7qffAR1Ytv8gV+YTv+cCtcNy8bxJ3KOURKQNW0jgnWNayzFpjrBI0GQOIk2 nVQpK4NZv9FmaU6xpzQScf83kP2RXQQfbK8k3zxlkRgohPBDZFAkyB+WSshoUw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643062823; a=rsa-sha256; cv=none; b=OoAGZ0iuP9kCfg1Q3lnEs5N0H2XVTgIQe/6bH4x8aznfPLzD3k4MGbPdB4kRdSiKp/TrMA YfQqPPtrbR2dVOXCnt8JWGn5K45qC1GAVRfzFiQBL1CDCgnaaFbzga5dCyyPMVTz9DRlBf A/IZz2QjNcL93mObnjgEwlY2Qlea8ae4XURVfwlgwFvj7ijMQcN4ODvLAgEs/OLaltXZJK trF25DI34BGvdzO5zv+MJDDqZHJF/PsJhRPJF4EOZpq1nxUghqVSpMSNBfFvdxcRLP67+t /PMVO8Ou642hcOS4LnBAoDveytSXdUGrcnaKM9Yz1ttagVnSb8NIMFiJ6PPRDA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=a8c4147edcdce934f93dd848c6ed083500dff22c commit a8c4147edcdce934f93dd848c6ed083500dff22c Author: John Baldwin AuthorDate: 2022-01-22 01:02:16 +0000 Commit: John Baldwin CommitDate: 2022-01-24 22:20:02 +0000 cxgbei: Parse all PDUs received prior to enabling offload mode. Previously this would only handle a single PDU that did not contain any data. This should now handle an arbitrary number of PDUs. While here check for these PDUs in the T6-specific CPL_RX_ISCSI_CMP handler in addition to CPL_RX_ISCSI_DDP. Reported by: Jithesh Arakkan @ Chelsio Sponsored by: Chelsio Communications --- sys/dev/cxgbe/cxgbei/cxgbei.c | 183 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 158 insertions(+), 25 deletions(-) diff --git a/sys/dev/cxgbe/cxgbei/cxgbei.c b/sys/dev/cxgbe/cxgbei/cxgbei.c index d6bf3ab871e8..bca21d211abd 100644 --- a/sys/dev/cxgbe/cxgbei/cxgbei.c +++ b/sys/dev/cxgbe/cxgbei/cxgbei.c @@ -43,6 +43,7 @@ __FBSDID("$FreeBSD$"); #ifdef TCP_OFFLOAD #include +#include #include #include #include @@ -299,6 +300,136 @@ do_rx_iscsi_data(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m return (0); } +static int +mbuf_crc32c_helper(void *arg, void *data, u_int len) +{ + uint32_t *digestp = arg; + + *digestp = calculate_crc32c(*digestp, data, len); + return (0); +} + +static bool +parse_pdus(struct toepcb *toep, struct icl_cxgbei_conn *icc, struct sockbuf *sb) +{ + struct iscsi_bhs bhs; + struct mbuf *m; + struct icl_pdu *ip; + u_int ahs_len, data_len, header_len, pdu_len, total_len; + uint32_t calc_digest, wire_digest; + + total_len = sbused(sb); + CTR3(KTR_CXGBE, "%s: tid %u, %u bytes in so_rcv", __func__, toep->tid, + total_len); + + m = sbcut_locked(sb, total_len); + KASSERT(m_length(m, NULL) == total_len, + ("sbcut returned less data (%u vs %u)", total_len, + m_length(m, NULL))); + + header_len = sizeof(struct iscsi_bhs); + if (icc->ic.ic_header_crc32c) + header_len += ISCSI_HEADER_DIGEST_SIZE; + for (;;) { + if (total_len < sizeof(struct iscsi_bhs)) { + ICL_WARN("truncated pre-offload PDU with len %u", + total_len); + m_freem(m); + return (false); + } + m_copydata(m, 0, sizeof(struct iscsi_bhs), (caddr_t)&bhs); + + ahs_len = bhs.bhs_total_ahs_len * 4; + data_len = bhs.bhs_data_segment_len[0] << 16 | + bhs.bhs_data_segment_len[1] << 8 | + bhs.bhs_data_segment_len[0]; + pdu_len = header_len + ahs_len + roundup2(data_len, 4); + if (icc->ic.ic_data_crc32c && data_len != 0) + pdu_len += ISCSI_DATA_DIGEST_SIZE; + + if (total_len < pdu_len) { + ICL_WARN("truncated pre-offload PDU len %u vs %u", + total_len, pdu_len); + m_freem(m); + return (false); + } + + if (ahs_len != 0) { + ICL_WARN("received pre-offload PDU with AHS"); + m_freem(m); + return (false); + } + + if (icc->ic.ic_header_crc32c) { + m_copydata(m, sizeof(struct iscsi_bhs), + sizeof(wire_digest), (caddr_t)&wire_digest); + + calc_digest = calculate_crc32c(0xffffffff, + (caddr_t)&bhs, sizeof(bhs)); + calc_digest ^= 0xffffffff; + if (calc_digest != wire_digest) { + ICL_WARN("received pre-offload PDU 0x%02x " + "with invalid header digest (0x%x vs 0x%x)", + bhs.bhs_opcode, wire_digest, calc_digest); + toep->ofld_rxq->rx_iscsi_header_digest_errors++; + m_free(m); + return (false); + } + } + + m_adj(m, header_len); + + if (icc->ic.ic_data_crc32c && data_len != 0) { + m_copydata(m, data_len, sizeof(wire_digest), + (caddr_t)&wire_digest); + + calc_digest = 0xffffffff; + m_apply(m, 0, roundup2(data_len, 4), mbuf_crc32c_helper, + &calc_digest); + calc_digest ^= 0xffffffff; + if (calc_digest != wire_digest) { + ICL_WARN("received pre-offload PDU 0x%02x " + "with invalid data digest (0x%x vs 0x%x)", + bhs.bhs_opcode, wire_digest, calc_digest); + toep->ofld_rxq->rx_iscsi_data_digest_errors++; + m_free(m); + return (false); + } + } + + ip = icl_cxgbei_new_pdu(M_NOWAIT); + if (ip == NULL) + CXGBE_UNIMPLEMENTED("PDU allocation failure"); + icl_cxgbei_new_pdu_set_conn(ip, &icc->ic); + *ip->ip_bhs = bhs; + ip->ip_data_len = data_len; + if (data_len != 0) + ip->ip_data_mbuf = m; + + STAILQ_INSERT_TAIL(&icc->rcvd_pdus, ip, ip_next); + + total_len -= pdu_len; + if (total_len == 0) { + if (data_len == 0) + m_freem(m); + return (true); + } + + if (data_len != 0) { + m = m_split(m, roundup2(data_len, 4), M_NOWAIT); + if (m == NULL) { + ICL_WARN("failed to split mbuf chain for " + "pre-offload PDU"); + + /* Don't free the mbuf chain as 'ip' owns it. */ + return (false); + } + if (icc->ic.ic_data_crc32c) + m_adj(m, ISCSI_DATA_DIGEST_SIZE); + } + } +} + static int do_rx_iscsi_ddp(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) { @@ -419,39 +550,24 @@ do_rx_iscsi_ddp(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) ic->ic_error(ic); return (0); } - icl_cxgbei_new_pdu_set_conn(ip, ic); - - MPASS(m == NULL); /* was unused, we'll use it now. */ - m = sbcut_locked(sb, sbused(sb)); /* XXXNP: toep->sb_cc accounting? */ - if (__predict_false(m != NULL)) { - int len = m_length(m, NULL); + if (__predict_false(sbused(sb)) != 0) { /* * PDUs were received before the tid transitioned to ULP mode. * Convert them to icl_cxgbei_pdus and send them to ICL before * the PDU in icp/ip. */ - CTR3(KTR_CXGBE, "%s: tid %u, %u bytes in so_rcv", __func__, tid, - len); - - /* XXXNP: needs to be rewritten. */ - if (len == sizeof(struct iscsi_bhs) || len == 4 + sizeof(struct - iscsi_bhs)) { - struct icl_cxgbei_pdu *icp0; - struct icl_pdu *ip0; - - ip0 = icl_cxgbei_new_pdu(M_NOWAIT); - if (ip0 == NULL) - CXGBE_UNIMPLEMENTED("PDU allocation failure"); - icl_cxgbei_new_pdu_set_conn(ip0, ic); - icp0 = ip_to_icp(ip0); - icp0->icp_seq = 0; /* XXX */ - icp0->icp_flags = ICPF_RX_HDR | ICPF_RX_STATUS; - m_copydata(m, 0, sizeof(struct iscsi_bhs), (void *)ip0->ip_bhs); - STAILQ_INSERT_TAIL(&icc->rcvd_pdus, ip0, ip_next); + if (!parse_pdus(toep, icc, sb)) { + SOCKBUF_UNLOCK(sb); + INP_WUNLOCK(inp); + + icl_cxgbei_conn_pdu_free(NULL, ip); + toep->ulpcb2 = NULL; + ic->ic_error(ic); + return (0); } - m_freem(m); } + icl_cxgbei_new_pdu_set_conn(ip, ic); STAILQ_INSERT_TAIL(&icc->rcvd_pdus, ip, ip_next); if ((icc->rx_flags & RXF_ACTIVE) == 0) { @@ -700,6 +816,23 @@ do_rx_iscsi_cmp(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) m_freem(m); return (0); } + + if (__predict_false(sbused(sb)) != 0) { + /* + * PDUs were received before the tid transitioned to ULP mode. + * Convert them to icl_cxgbei_pdus and send them to ICL before + * the PDU in icp/ip. + */ + if (!parse_pdus(toep, icc, sb)) { + SOCKBUF_UNLOCK(sb); + INP_WUNLOCK(inp); + + icl_cxgbei_conn_pdu_free(NULL, ip); + toep->ulpcb2 = NULL; + ic->ic_error(ic); + return (0); + } + } icl_cxgbei_new_pdu_set_conn(ip, ic); /* Enqueue the PDU to the received pdus queue. */