From nobody Fri Jun 23 15:09:55 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qngdb6sCfz4gjfG; Fri, 23 Jun 2023 15:09:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qngdb624zz3PY6; Fri, 23 Jun 2023 15:09:55 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687532995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4CEOKnh1qSqa4E5E6G7+lTDRiy7y7BlQsu7Gj/rDWIA=; b=lPrmSpsr3ehJTmJA0UfHKlMYLChBcuDxztt8ZxoMWXcoN3mgPP4FrWgqql0PSsPPXebiMa 6YdFL1CXOVQ/Ug0fR530qErIt0BMHh0b4FmHOJSx7JmbLDlZCgSwYCpWkersG8OIlxrHGg /5rww2EU6HV5M2k8ngxKhlC7kFfZ8XtTfr/aA6mOOZBD3fbZNTmrMxFq5PJ4fOnoxuokH8 X1HndIguug1k3O8DRD6vqlVbUX0gQ7SMCilFPJlN9u2z221cN9nGq1lE++RITTkoVX/ToG uyDa/NnEMQE6FX5nqkYZ0TvYMHlZojXxjPyR/ByzRJMhA56zWXo0MRMjm5S3RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687532995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4CEOKnh1qSqa4E5E6G7+lTDRiy7y7BlQsu7Gj/rDWIA=; b=aHbD7+vYGM1L23s26KGLLxNCmy3AHauRMwyK7nhh5H4ePoViAbkTQoW7GimtAdpU0OCxW0 LFfAZlJLxRbvLYOxscm3Pr8KMap37ZFGI3omc4bNXSNbWRSlSkwgPElblWsr+BWXe38uai fC1CyyexHTtogiPbSoq+41bEtdAKJ3HlL3JIC0416sG9kflpMx/ank9zVzViG71rDlGjeF m4GUDQlVe0jqRllYZ8cSAqnar+wU5DkdzsKUwG1Lgq6Gb2AK8/XcwhblJx/VxN7P7Y+Jqj 6bF8RLiwsx70pOlNAI16paT4co/MwZKWYVQSxw2e8ki3dP/aAj4O5oq44LROKA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1687532995; a=rsa-sha256; cv=none; b=fIO2MdQ+M2wDXES9sQGu3YzVEb3CNN/kyG00vWaXOvg1e8koYpGIt9awaHqcYmExZl2Ip5 nsq+PrCMZcQDGSGfYY+YUu1M0jjVATukMO56WuGoUuWnowbv2mGz/wzahydH9OmlQAufDZ ofb8AXtQmUax+ezotZtXy8aCWdLF9PmwwoJflVpc64azgdqg29o3b9OBHPtF3iCwWpUwBs C6jz8YZ28Sg4jk04s4RIrvL9Gbd+RojropIQM7cLneT6TK15dcjCskUBoaABV3ybD1yXbi HRhSsf09aGftD/BxWQfVdjwMGwQlmkvaiUa/x1IJoPrr7QTP33FQVsUPHYFqxA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Qngdb57Zvz17vq; Fri, 23 Jun 2023 15:09:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 35NF9tgd037746; Fri, 23 Jun 2023 15:09:55 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 35NF9t5v037745; Fri, 23 Jun 2023 15:09:55 GMT (envelope-from git) Date: Fri, 23 Jun 2023 15:09:55 GMT Message-Id: <202306231509.35NF9t5v037745@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: 6775ef4188b4 - main - netinet6: Implement in6_cksum_partial() using m_apply() List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 6775ef4188b4d4c023e76ebd2b71fa8c2c7e7cd2 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=6775ef4188b4d4c023e76ebd2b71fa8c2c7e7cd2 commit 6775ef4188b4d4c023e76ebd2b71fa8c2c7e7cd2 Author: Mark Johnston AuthorDate: 2023-06-23 13:55:43 +0000 Commit: Mark Johnston CommitDate: 2023-06-23 13:55:43 +0000 netinet6: Implement in6_cksum_partial() using m_apply() This ensures that in6_cksum_partial() can be applied to unmapped mbufs, which can happen at least when icmp6_reflect() quotes a packet. The basic idea is to restructure in6_cksum_partial() to operate on one mbuf at a time. If the buffer length is odd or unaligned, an extra residual byte may be returned, to be incorporated into the checksum when processing the next buffer. PR: 268400 Reviewed by: cy MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40598 --- sys/netinet6/in6.h | 6 +- sys/netinet6/in6_cksum.c | 300 +++++++++++++++++++++-------------------------- 2 files changed, 139 insertions(+), 167 deletions(-) diff --git a/sys/netinet6/in6.h b/sys/netinet6/in6.h index 41fe07fe093d..c15b2485964a 100644 --- a/sys/netinet6/in6.h +++ b/sys/netinet6/in6.h @@ -659,10 +659,10 @@ struct ip6_mtuinfo { struct cmsghdr; struct ip6_hdr; +int in6_cksum(struct mbuf *, uint8_t, uint32_t, uint32_t); +int in6_cksum_partial(struct mbuf *, uint8_t, uint32_t, uint32_t, uint32_t); int in6_cksum_pseudo(struct ip6_hdr *, uint32_t, uint8_t, uint16_t); -int in6_cksum(struct mbuf *, u_int8_t, u_int32_t, u_int32_t); -int in6_cksum_partial(struct mbuf *, u_int8_t, u_int32_t, u_int32_t, - u_int32_t); + int in6_localaddr(struct in6_addr *); int in6_localip(struct in6_addr *); bool in6_localip_fib(struct in6_addr *, uint16_t); diff --git a/sys/netinet6/in6_cksum.c b/sys/netinet6/in6_cksum.c index c4624b90b6a1..a93db45eb968 100644 --- a/sys/netinet6/in6_cksum.c +++ b/sys/netinet6/in6_cksum.c @@ -82,6 +82,16 @@ __FBSDID("$FreeBSD$"); #define ADDCARRY(x) (x > 65535 ? x -= 65535 : x) #define REDUCE {l_util.l = sum; sum = l_util.s[0] + l_util.s[1]; (void)ADDCARRY(sum);} +union l_util { + uint16_t s[2]; + uint32_t l; +}; + +union s_util { + uint8_t c[2]; + uint16_t s; +}; + static int _in6_cksum_pseudo(struct ip6_hdr *ip6, uint32_t len, uint8_t nxt, uint16_t csum) { @@ -131,17 +141,115 @@ _in6_cksum_pseudo(struct ip6_hdr *ip6, uint32_t len, uint8_t nxt, uint16_t csum) int in6_cksum_pseudo(struct ip6_hdr *ip6, uint32_t len, uint8_t nxt, uint16_t csum) { + union l_util l_util; int sum; - union { - u_int16_t s[2]; - u_int32_t l; - } l_util; sum = _in6_cksum_pseudo(ip6, len, nxt, csum); REDUCE; return (sum); } +static int +in6_cksumdata(void *data, int *lenp, uint8_t *residp, int rlen) +{ + union l_util l_util; + union s_util s_util; + uint16_t *w; + int len, sum; + bool byte_swapped; + + KASSERT(*lenp >= 0, ("%s: negative len %d", __func__, *lenp)); + KASSERT(rlen == 0 || rlen == 1, ("%s: rlen %d", __func__, rlen)); + + len = *lenp; + sum = 0; + + if (len == 0) { + len = rlen; + goto out; + } + + byte_swapped = false; + w = data; + + /* + * Do we have a residual byte left over from the previous buffer? + */ + if (rlen == 1) { + s_util.c[0] = *residp; + s_util.c[1] = *(uint8_t *)w; + sum += s_util.s; + w = (uint16_t *)((uint8_t *)w + 1); + len--; + rlen = 0; + } + + /* + * Force to even boundary. + */ + if ((1 & (uintptr_t)w) && len > 0) { + REDUCE; + sum <<= 8; + s_util.c[0] = *(uint8_t *)w; + w = (uint16_t *)((uint8_t *)w + 1); + len--; + byte_swapped = true; + } + + /* + * Unroll the loop to make overhead from branches &c small. + */ + while ((len -= 32) >= 0) { + sum += w[0]; sum += w[1]; sum += w[2]; sum += w[3]; + sum += w[4]; sum += w[5]; sum += w[6]; sum += w[7]; + sum += w[8]; sum += w[9]; sum += w[10]; sum += w[11]; + sum += w[12]; sum += w[13]; sum += w[14]; sum += w[15]; + w += 16; + } + len += 32; + while ((len -= 8) >= 0) { + sum += w[0]; sum += w[1]; sum += w[2]; sum += w[3]; + w += 4; + } + len += 8; + if (len == 0 && !byte_swapped) + goto out; + REDUCE; + while ((len -= 2) >= 0) { + sum += *w++; + } + if (byte_swapped) { + REDUCE; + sum <<= 8; + if (len == -1) { + s_util.c[1] = *(uint8_t *)w; + sum += s_util.s; + } else /* len == -2 */ + *residp = s_util.c[0]; + len++; + } else if (len == -1) + *residp = *(uint8_t *)w; +out: + *lenp = len & 1; + return (sum); +} + +struct in6_cksum_partial_arg { + int sum; + int rlen; + uint8_t resid; +}; + +static int +in6_cksum_partial_one(void *_arg, void *data, u_int len) +{ + struct in6_cksum_partial_arg *arg = _arg; + + arg->sum += in6_cksumdata(data, &len, &arg->resid, arg->rlen); + arg->rlen = len; + return (0); +} + /* * m MUST contain a contiguous IP6 header. * off is an offset where TCP/UDP/ICMP6 header starts. @@ -150,33 +258,29 @@ in6_cksum_pseudo(struct ip6_hdr *ip6, uint32_t len, uint8_t nxt, uint16_t csum) * cov is the number of bytes to be taken into account for the checksum */ int -in6_cksum_partial(struct mbuf *m, u_int8_t nxt, u_int32_t off, - u_int32_t len, u_int32_t cov) +in6_cksum_partial(struct mbuf *m, uint8_t nxt, uint32_t off, uint32_t len, + uint32_t cov) { + struct in6_cksum_partial_arg arg; + union l_util l_util; + union s_util s_util; struct ip6_hdr *ip6; - u_int16_t *w, scope; - int byte_swapped, mlen; + uint16_t *w, scope; int sum; union { - u_int16_t phs[4]; + uint16_t phs[4]; struct { - u_int32_t ph_len; - u_int8_t ph_zero[3]; - u_int8_t ph_nxt; + uint32_t ph_len; + uint8_t ph_zero[3]; + uint8_t ph_nxt; } __packed ph; } uph; - union { - u_int8_t c[2]; - u_int16_t s; - } s_util; - union { - u_int16_t s[2]; - u_int32_t l; - } l_util; /* Sanity check. */ KASSERT(m->m_pkthdr.len >= off + len, ("%s: mbuf len (%d) < off(%d)+" "len(%d)", __func__, m->m_pkthdr.len, off, len)); + KASSERT(m->m_len >= sizeof(*ip6), + ("%s: mbuf len %d < sizeof(ip6)", __func__, m->m_len)); /* * First create IP6 pseudo header and calculate a summary. @@ -193,7 +297,7 @@ in6_cksum_partial(struct mbuf *m, u_int8_t nxt, u_int32_t off, /* IPv6 source address. */ scope = in6_getscope(&ip6->ip6_src); - w = (u_int16_t *)&ip6->ip6_src; + w = (uint16_t *)&ip6->ip6_src; sum += w[0]; sum += w[1]; sum += w[2]; sum += w[3]; sum += w[4]; sum += w[5]; sum += w[6]; sum += w[7]; if (scope != 0) @@ -201,158 +305,26 @@ in6_cksum_partial(struct mbuf *m, u_int8_t nxt, u_int32_t off, /* IPv6 destination address. */ scope = in6_getscope(&ip6->ip6_dst); - w = (u_int16_t *)&ip6->ip6_dst; + w = (uint16_t *)&ip6->ip6_dst; sum += w[0]; sum += w[1]; sum += w[2]; sum += w[3]; sum += w[4]; sum += w[5]; sum += w[6]; sum += w[7]; if (scope != 0) sum -= scope; /* - * Secondly calculate a summary of the first mbuf excluding offset. + * Loop over the rest of the mbuf chain and compute the rest of the + * checksum. m_apply() handles unmapped mbufs. */ - while (off > 0) { - if (m->m_len <= off) - off -= m->m_len; - else - break; - m = m->m_next; - } - w = (u_int16_t *)(mtod(m, u_char *) + off); - mlen = m->m_len - off; - if (cov < mlen) - mlen = cov; - cov -= mlen; - /* - * Force to even boundary. - */ - if ((1 & (long)w) && (mlen > 0)) { - REDUCE; - sum <<= 8; - s_util.c[0] = *(u_char *)w; - w = (u_int16_t *)((char *)w + 1); - mlen--; - byte_swapped = 1; - } else - byte_swapped = 0; + arg.sum = sum; + arg.rlen = 0; + (void)m_apply(m, off, cov, in6_cksum_partial_one, &arg); + sum = arg.sum; /* - * Unroll the loop to make overhead from - * branches &c small. + * Handle a residual byte. */ - while ((mlen -= 32) >= 0) { - sum += w[0]; sum += w[1]; sum += w[2]; sum += w[3]; - sum += w[4]; sum += w[5]; sum += w[6]; sum += w[7]; - sum += w[8]; sum += w[9]; sum += w[10]; sum += w[11]; - sum += w[12]; sum += w[13]; sum += w[14]; sum += w[15]; - w += 16; - } - mlen += 32; - while ((mlen -= 8) >= 0) { - sum += w[0]; sum += w[1]; sum += w[2]; sum += w[3]; - w += 4; - } - mlen += 8; - if (mlen == 0 && byte_swapped == 0) - goto next; - REDUCE; - while ((mlen -= 2) >= 0) { - sum += *w++; - } - if (byte_swapped) { - REDUCE; - sum <<= 8; - byte_swapped = 0; - if (mlen == -1) { - s_util.c[1] = *(char *)w; - sum += s_util.s; - mlen = 0; - } else - mlen = -1; - } else if (mlen == -1) - s_util.c[0] = *(char *)w; - next: - m = m->m_next; - - /* - * Lastly calculate a summary of the rest of mbufs. - */ - - for (;m && cov; m = m->m_next) { - if (m->m_len == 0) - continue; - w = mtod(m, u_int16_t *); - if (mlen == -1) { - /* - * The first byte of this mbuf is the continuation - * of a word spanning between this mbuf and the - * last mbuf. - * - * s_util.c[0] is already saved when scanning previous - * mbuf. - */ - s_util.c[1] = *(char *)w; - sum += s_util.s; - w = (u_int16_t *)((char *)w + 1); - mlen = m->m_len - 1; - cov--; - } else - mlen = m->m_len; - if (cov < mlen) - mlen = cov; - cov -= mlen; - /* - * Force to even boundary. - */ - if ((1 & (long) w) && (mlen > 0)) { - REDUCE; - sum <<= 8; - s_util.c[0] = *(u_char *)w; - w = (u_int16_t *)((char *)w + 1); - mlen--; - byte_swapped = 1; - } - /* - * Unroll the loop to make overhead from - * branches &c small. - */ - while ((mlen -= 32) >= 0) { - sum += w[0]; sum += w[1]; sum += w[2]; sum += w[3]; - sum += w[4]; sum += w[5]; sum += w[6]; sum += w[7]; - sum += w[8]; sum += w[9]; sum += w[10]; sum += w[11]; - sum += w[12]; sum += w[13]; sum += w[14]; sum += w[15]; - w += 16; - } - mlen += 32; - while ((mlen -= 8) >= 0) { - sum += w[0]; sum += w[1]; sum += w[2]; sum += w[3]; - w += 4; - } - mlen += 8; - if (mlen == 0 && byte_swapped == 0) - continue; - REDUCE; - while ((mlen -= 2) >= 0) { - sum += *w++; - } - if (byte_swapped) { - REDUCE; - sum <<= 8; - byte_swapped = 0; - if (mlen == -1) { - s_util.c[1] = *(char *)w; - sum += s_util.s; - mlen = 0; - } else - mlen = -1; - } else if (mlen == -1) - s_util.c[0] = *(char *)w; - } - if (cov) - panic("in6_cksum: out of data"); - if (mlen == -1) { - /* The last mbuf has odd # of bytes. Follow the - standard (the odd byte may be shifted left by 8 bits - or not as determined by endian-ness of the machine) */ + if (arg.rlen == 1) { + s_util.c[0] = arg.resid; s_util.c[1] = 0; sum += s_util.s; } @@ -361,7 +333,7 @@ in6_cksum_partial(struct mbuf *m, u_int8_t nxt, u_int32_t off, } int -in6_cksum(struct mbuf *m, u_int8_t nxt, u_int32_t off, u_int32_t len) +in6_cksum(struct mbuf *m, uint8_t nxt, uint32_t off, uint32_t len) { return (in6_cksum_partial(m, nxt, off, len, len)); }