From nobody Thu Dec 09 20:17:47 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BE5E918DA6A9; Thu, 9 Dec 2021 20:17:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J951l6vlrz4qv2; Thu, 9 Dec 2021 20:17:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 32C37511F; Thu, 9 Dec 2021 20:17:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1B9KHlZ0053831; Thu, 9 Dec 2021 20:17:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1B9KHlxU053830; Thu, 9 Dec 2021 20:17:47 GMT (envelope-from git) Date: Thu, 9 Dec 2021 20:17:47 GMT Message-Id: <202112092017.1B9KHlxU053830@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 6113a08b98e4 - main - cryptosoft: Fully support per-operation keys for auth algorithms. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 6113a08b98e403de5b92cc0a30fdc60489eccc48 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1639081068; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=rzn4nFBHDpKYVT2Tz+UAbGE8AREzVC9XFzQSOem6AtU=; b=f7kU4YuvqbKlyi8oRUJEOWbltoVPt8wb7cIwK5OenIaX7+0l2zkqWHQl/Ezo0DUPmBN4OA pNsIR5x7kSIKPqVmR0ejr8iNh8WexXc93q8bVK/TbTFcC0ycwOZW+SFHCvTR9ZnVbh/6dt ZldRKqVfOgI2x+jkMhRNWeDvOd8p4gkC1bY9fgSROBSa+RvKlksfr1E7E5Bcharrx40Mg8 53BH7hxUHsM4IvrEKuy9zyjwQSZ+qC2dVIshUGtdT/NZJ1agtnruRn9LMETK+2/C8IBwsp YhMzj4lMjdnZ+UGWVD03RKlS8QlCngcCUwJ3xcbw7y9NGTRwu+Ru4XFKB6bJ8g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1639081068; a=rsa-sha256; cv=none; b=HnvOedizbZME/phnW5oNMLPQ2QwYedYU8Td/tnniGpQTgJhZobVRy9/Cw3nslKxIv/zk+p LHm8aSk/590MDv00dUgpSUONVas5byE1pTJei9WWWAzN9EVjNTc8Y6Qxfu3guFgBChCfWD EgRLEKUN9Sk8YtVGnl9gzvERl8IzRqKSOQxmxIK065/CNYFIknQJ9J3z98cZtX6u6l/8hx RmizZCegyHd8Ba3biIBo+id6+F+ecD8NHgzuXTRCN8EDjG0lK4YILI0i9raYPQg/fGkHI5 MRQBAE8xdWl6P8tsuBUsJ/XGU1S8NcMi30XGhIAxxOv3jdaJLF7zQC1q8WyvAw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=6113a08b98e403de5b92cc0a30fdc60489eccc48 commit 6113a08b98e403de5b92cc0a30fdc60489eccc48 Author: John Baldwin AuthorDate: 2021-12-09 19:52:42 +0000 Commit: John Baldwin CommitDate: 2021-12-09 19:52:42 +0000 cryptosoft: Fully support per-operation keys for auth algorithms. Only pre-allocate auth contexts when a session-wide key is provided or for sessions without keys. For sessions with per-operation keys, always initialize the on-stack context directly rather than initializing the session context in swcr_authprepare (now removed) and then copying that session context into the on-stack context. This approach permits parallel auth operations without needing a serializing lock. In addition, the previous code assumed that auth sessions always provided an initial key unlike cipher sessions which assume either an initial key or per-op keys. While here, fix the Blake2 auth transforms to function like other auth transforms where Setkey is invoked after Init rather than before. Reviewed by: markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D33316 --- sys/crypto/blake2/blake2-sw.c | 38 +++++------- sys/opencrypto/cryptosoft.c | 140 +++++++++++++++++++++--------------------- 2 files changed, 83 insertions(+), 95 deletions(-) diff --git a/sys/crypto/blake2/blake2-sw.c b/sys/crypto/blake2/blake2-sw.c index 449ef2be94f5..dafe0e3f84a4 100644 --- a/sys/crypto/blake2/blake2-sw.c +++ b/sys/crypto/blake2/blake2-sw.c @@ -28,8 +28,6 @@ extern int blake2s_ref(uint8_t *out, const void *in, const void *key, struct blake2b_xform_ctx { blake2b_state state; - uint8_t key[BLAKE2B_KEYBYTES]; - uint16_t klen; }; CTASSERT(sizeof(union authctx) >= sizeof(struct blake2b_xform_ctx)); @@ -39,24 +37,21 @@ blake2b_xform_init(void *vctx) struct blake2b_xform_ctx *ctx = vctx; int rc; - if (ctx->klen > 0) - rc = blake2b_init_key_ref(&ctx->state, BLAKE2B_OUTBYTES, - ctx->key, ctx->klen); - else - rc = blake2b_init_ref(&ctx->state, BLAKE2B_OUTBYTES); + rc = blake2b_init_ref(&ctx->state, BLAKE2B_OUTBYTES); if (rc != 0) - panic("blake2b_init_key: invalid arguments"); + panic("blake2b_init: invalid arguments"); } static void blake2b_xform_setkey(void *vctx, const uint8_t *key, u_int klen) { struct blake2b_xform_ctx *ctx = vctx; + int rc; - if (klen > sizeof(ctx->key)) - panic("invalid klen %u", (unsigned)klen); - memcpy(ctx->key, key, klen); - ctx->klen = klen; + rc = blake2b_init_key_ref(&ctx->state, BLAKE2B_OUTBYTES, key, + klen); + if (rc != 0) + panic("blake2b_init_key: invalid arguments"); } static int @@ -96,8 +91,6 @@ const struct auth_hash auth_hash_blake2b = { struct blake2s_xform_ctx { blake2s_state state; - uint8_t key[BLAKE2S_KEYBYTES]; - uint16_t klen; }; CTASSERT(sizeof(union authctx) >= sizeof(struct blake2s_xform_ctx)); @@ -107,24 +100,21 @@ blake2s_xform_init(void *vctx) struct blake2s_xform_ctx *ctx = vctx; int rc; - if (ctx->klen > 0) - rc = blake2s_init_key_ref(&ctx->state, BLAKE2S_OUTBYTES, - ctx->key, ctx->klen); - else - rc = blake2s_init_ref(&ctx->state, BLAKE2S_OUTBYTES); + rc = blake2s_init_ref(&ctx->state, BLAKE2S_OUTBYTES); if (rc != 0) - panic("blake2s_init_key: invalid arguments"); + panic("blake2s_init: invalid arguments"); } static void blake2s_xform_setkey(void *vctx, const uint8_t *key, u_int klen) { struct blake2s_xform_ctx *ctx = vctx; + int rc; - if (klen > sizeof(ctx->key)) - panic("invalid klen %u", (unsigned)klen); - memcpy(ctx->key, key, klen); - ctx->klen = klen; + rc = blake2s_init_key_ref(&ctx->state, BLAKE2S_OUTBYTES, key, + klen); + if (rc != 0) + panic("blake2s_init_key: invalid arguments"); } static int diff --git a/sys/opencrypto/cryptosoft.c b/sys/opencrypto/cryptosoft.c index b6bb7979c536..576efeaec20f 100644 --- a/sys/opencrypto/cryptosoft.c +++ b/sys/opencrypto/cryptosoft.c @@ -65,6 +65,7 @@ struct swcr_auth { void *sw_octx; const struct auth_hash *sw_axf; uint16_t sw_mlen; + bool sw_hmac; }; struct swcr_encdec { @@ -283,33 +284,6 @@ swcr_encdec(struct swcr_session *ses, struct cryptop *crp) return (0); } -static void -swcr_authprepare(const struct auth_hash *axf, struct swcr_auth *sw, - const uint8_t *key, int klen) -{ - - switch (axf->type) { - case CRYPTO_SHA1_HMAC: - case CRYPTO_SHA2_224_HMAC: - case CRYPTO_SHA2_256_HMAC: - case CRYPTO_SHA2_384_HMAC: - case CRYPTO_SHA2_512_HMAC: - case CRYPTO_NULL_HMAC: - case CRYPTO_RIPEMD160_HMAC: - hmac_init_ipad(axf, key, klen, sw->sw_ictx); - hmac_init_opad(axf, key, klen, sw->sw_octx); - break; - case CRYPTO_POLY1305: - case CRYPTO_BLAKE2B: - case CRYPTO_BLAKE2S: - axf->Setkey(sw->sw_ictx, key, klen); - axf->Init(sw->sw_ictx); - break; - default: - panic("%s: algorithm %d doesn't use keys", __func__, axf->type); - } -} - /* * Compute or verify hash. */ @@ -318,7 +292,7 @@ swcr_authcompute(struct swcr_session *ses, struct cryptop *crp) { u_char aalg[HASH_MAX_LEN]; const struct crypto_session_params *csp; - struct swcr_auth *sw; + const struct swcr_auth *sw; const struct auth_hash *axf; union authctx ctx; int err; @@ -329,11 +303,16 @@ swcr_authcompute(struct swcr_session *ses, struct cryptop *crp) csp = crypto_get_params(crp->crp_session); if (crp->crp_auth_key != NULL) { - swcr_authprepare(axf, sw, crp->crp_auth_key, - csp->csp_auth_klen); - } - - bcopy(sw->sw_ictx, &ctx, axf->ctxsize); + if (sw->sw_hmac) { + hmac_init_ipad(axf, crp->crp_auth_key, + csp->csp_auth_klen, &ctx); + } else { + axf->Init(&ctx); + axf->Setkey(&ctx, crp->crp_auth_key, + csp->csp_auth_klen); + } + } else + memcpy(&ctx, sw->sw_ictx, axf->ctxsize); if (crp->crp_aad != NULL) err = axf->Update(&ctx, crp->crp_aad, crp->crp_aad_length); @@ -358,8 +337,12 @@ swcr_authcompute(struct swcr_session *ses, struct cryptop *crp) axf->Update(&ctx, crp->crp_esn, 4); axf->Final(aalg, &ctx); - if (sw->sw_octx != NULL) { - bcopy(sw->sw_octx, &ctx, axf->ctxsize); + if (sw->sw_hmac) { + if (crp->crp_auth_key != NULL) + hmac_init_opad(axf, crp->crp_auth_key, + csp->csp_auth_klen, &ctx); + else + memcpy(&ctx, sw->sw_octx, axf->ctxsize); axf->Update(&ctx, aalg, axf->hashsize); axf->Final(aalg, &ctx); } @@ -394,7 +377,7 @@ swcr_gmac(struct swcr_session *ses, struct cryptop *crp) struct crypto_buffer_cursor cc; const u_char *inblk; union authctx ctx; - struct swcr_auth *swa; + const struct swcr_auth *swa; const struct auth_hash *axf; uint32_t *blkp; size_t len; @@ -402,12 +385,17 @@ swcr_gmac(struct swcr_session *ses, struct cryptop *crp) swa = &ses->swcr_auth; axf = swa->sw_axf; - - bcopy(swa->sw_ictx, &ctx, axf->ctxsize); blksz = GMAC_BLOCK_LEN; KASSERT(axf->blocksize == blksz, ("%s: axf block size mismatch", __func__)); + if (crp->crp_auth_key != NULL) { + axf->Init(&ctx); + axf->Setkey(&ctx, crp->crp_auth_key, + crypto_get_params(crp->crp_session)->csp_auth_klen); + } else + memcpy(&ctx, swa->sw_ictx, axf->ctxsize); + /* Initialize the IV */ ivlen = AES_GCM_IV_LEN; crypto_read_iv(crp, iv); @@ -694,7 +682,7 @@ swcr_ccm_cbc_mac(struct swcr_session *ses, struct cryptop *crp) u_char tag[AES_CBC_MAC_HASH_LEN]; union authctx ctx; const struct crypto_session_params *csp; - struct swcr_auth *swa; + const struct swcr_auth *swa; const struct auth_hash *axf; int error, ivlen, len; @@ -702,7 +690,11 @@ swcr_ccm_cbc_mac(struct swcr_session *ses, struct cryptop *crp) swa = &ses->swcr_auth; axf = swa->sw_axf; - bcopy(swa->sw_ictx, &ctx, axf->ctxsize); + if (crp->crp_auth_key != NULL) { + axf->Init(&ctx); + axf->Setkey(&ctx, crp->crp_auth_key, csp->csp_auth_klen); + } else + memcpy(&ctx, swa->sw_ictx, axf->ctxsize); /* Initialize the IV */ ivlen = csp->csp_ivlen; @@ -1218,9 +1210,12 @@ swcr_setup_auth(struct swcr_session *ses, swa->sw_mlen = axf->hashsize; else swa->sw_mlen = csp->csp_auth_mlen; - swa->sw_ictx = malloc(axf->ctxsize, M_CRYPTO_DATA, M_NOWAIT); - if (swa->sw_ictx == NULL) - return (ENOBUFS); + if (csp->csp_auth_klen == 0 || csp->csp_auth_key != NULL) { + swa->sw_ictx = malloc(axf->ctxsize, M_CRYPTO_DATA, + M_NOWAIT); + if (swa->sw_ictx == NULL) + return (ENOBUFS); + } switch (csp->csp_auth_alg) { case CRYPTO_SHA1_HMAC: @@ -1230,18 +1225,17 @@ swcr_setup_auth(struct swcr_session *ses, case CRYPTO_SHA2_512_HMAC: case CRYPTO_NULL_HMAC: case CRYPTO_RIPEMD160_HMAC: - swa->sw_octx = malloc(axf->ctxsize, M_CRYPTO_DATA, - M_NOWAIT); - if (swa->sw_octx == NULL) - return (ENOBUFS); - + swa->sw_hmac = true; if (csp->csp_auth_key != NULL) { - swcr_authprepare(axf, swa, csp->csp_auth_key, - csp->csp_auth_klen); + swa->sw_octx = malloc(axf->ctxsize, M_CRYPTO_DATA, + M_NOWAIT); + if (swa->sw_octx == NULL) + return (ENOBUFS); + hmac_init_ipad(axf, csp->csp_auth_key, + csp->csp_auth_klen, swa->sw_ictx); + hmac_init_opad(axf, csp->csp_auth_key, + csp->csp_auth_klen, swa->sw_octx); } - - if (csp->csp_mode == CSP_MODE_DIGEST) - ses->swcr_process = swcr_authcompute; break; case CRYPTO_SHA1: case CRYPTO_SHA2_224: @@ -1249,37 +1243,41 @@ swcr_setup_auth(struct swcr_session *ses, case CRYPTO_SHA2_384: case CRYPTO_SHA2_512: axf->Init(swa->sw_ictx); - if (csp->csp_mode == CSP_MODE_DIGEST) - ses->swcr_process = swcr_authcompute; break; case CRYPTO_AES_NIST_GMAC: - axf->Init(swa->sw_ictx); - axf->Setkey(swa->sw_ictx, csp->csp_auth_key, - csp->csp_auth_klen); - if (csp->csp_mode == CSP_MODE_DIGEST) - ses->swcr_process = swcr_gmac; - break; + case CRYPTO_AES_CCM_CBC_MAC: case CRYPTO_POLY1305: + if (csp->csp_auth_key != NULL) { + axf->Init(swa->sw_ictx); + axf->Setkey(swa->sw_ictx, csp->csp_auth_key, + csp->csp_auth_klen); + } + break; case CRYPTO_BLAKE2B: case CRYPTO_BLAKE2S: /* * Blake2b and Blake2s support an optional key but do * not require one. */ - if (csp->csp_auth_klen == 0 || csp->csp_auth_key != NULL) + if (csp->csp_auth_klen == 0) + axf->Init(swa->sw_ictx); + else if (csp->csp_auth_key != NULL) axf->Setkey(swa->sw_ictx, csp->csp_auth_key, csp->csp_auth_klen); - axf->Init(swa->sw_ictx); - if (csp->csp_mode == CSP_MODE_DIGEST) - ses->swcr_process = swcr_authcompute; break; - case CRYPTO_AES_CCM_CBC_MAC: - axf->Init(swa->sw_ictx); - axf->Setkey(swa->sw_ictx, csp->csp_auth_key, - csp->csp_auth_klen); - if (csp->csp_mode == CSP_MODE_DIGEST) + } + + if (csp->csp_mode == CSP_MODE_DIGEST) { + switch (csp->csp_auth_alg) { + case CRYPTO_AES_NIST_GMAC: + ses->swcr_process = swcr_gmac; + break; + case CRYPTO_AES_CCM_CBC_MAC: ses->swcr_process = swcr_ccm_cbc_mac; - break; + break; + default: + ses->swcr_process = swcr_authcompute; + } } return (0);