From nobody Tue Jan 11 22:38:16 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DB4E3194C624; Tue, 11 Jan 2022 22:38:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JYQZd0WtQz4mKL; Tue, 11 Jan 2022 22:38:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 51837219BB; Tue, 11 Jan 2022 22:38:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20BMcGTY075984; Tue, 11 Jan 2022 22:38:16 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20BMcG0I075983; Tue, 11 Jan 2022 22:38:16 GMT (envelope-from git) Date: Tue, 11 Jan 2022 22:38:16 GMT Message-Id: <202201112238.20BMcG0I075983@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 5022c68732e6 - main - cryptosoft: Use multi-block encrypt/decrypt for ChaCha20-Poly1305. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 5022c68732e6afadf39bca3fe3e65ddc44410e84 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1641940698; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bJbkQIKcdnI73st3eEaITf6vv3L6hLj6lQcarqBP+aY=; b=JK8e9TAyGhE3Tq8doW9LFvt0X/2p4oHhkHnHZHN2SHz368jHwUlvtJZFKpeSwNxXnJ2iXn cQTVJAIXwXICu9LMoH0YyYf79KAotNMhEXxwqG8bf5CxJSY2kxhJK0uxO0k62usc8aNCej VNzQDCBYXBMnsmodBPz/YtoS4m4Ub382/Bw/rqEbkINwfDXfb9/P4WxJ3aBEwhMbxGHBqt ODPURPdh1y7FXiZBko8rUqZYJSOUrfVpcSdasIEq7nD7qKHTLpMBTirGEHPlb7EBEiSAE1 jRKHq73d+EtJqmtui1rYRqPYgRUCT5y+l6jl5sAsEw+Ih8/trhJyQp7GTuYclg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1641940698; a=rsa-sha256; cv=none; b=NHjvxef0DdLDe1Gv71bfhfuTKCmCzLDB8VB/n2Vc1OfiOVzbsG8/HIizpd+EJSaWy/txq6 1H/p91+ZjJkHAmyrZhXx91qESZnAtt/Gy/8B4oxOT5vL5XopM1aSctR/b/m2bJzbyBRMww UxQurwqyNzuvebOSDfvOmqfEzLPDOBY7h6UdnN04nmA9HGqH7nutT7ILOSFul/oxJZ7k41 PfxKImwqSwTts4b3B+RJD2Tx6JJ+sabk1Xw3HrTt5SoauiZcpq5dJKrBCmWKk08KxG7Med D57d7CO0r+/SaAa3DfU5m4PiMsftjkL0PZh6aqv7CH8dd2o0YCRBZ9XxHAtg7w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=5022c68732e6afadf39bca3fe3e65ddc44410e84 commit 5022c68732e6afadf39bca3fe3e65ddc44410e84 Author: John Baldwin AuthorDate: 2022-01-11 22:21:31 +0000 Commit: John Baldwin CommitDate: 2022-01-11 22:21:31 +0000 cryptosoft: Use multi-block encrypt/decrypt for ChaCha20-Poly1305. Reviewed by: markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D33758 --- sys/opencrypto/cryptosoft.c | 132 ++++++++++++++++++++++++++++++-------------- 1 file changed, 92 insertions(+), 40 deletions(-) diff --git a/sys/opencrypto/cryptosoft.c b/sys/opencrypto/cryptosoft.c index 93b2c76cde4e..4d0f7d8718cc 100644 --- a/sys/opencrypto/cryptosoft.c +++ b/sys/opencrypto/cryptosoft.c @@ -974,12 +974,12 @@ swcr_chacha20_poly1305(const struct swcr_session *ses, struct cryptop *crp) struct crypto_buffer_cursor cc_in, cc_out; const u_char *inblk; u_char *outblk; + size_t inlen, outlen, todo; uint64_t *blkp; const struct swcr_auth *swa; const struct swcr_encdec *swe; const struct enc_xform *exf; void *ctx; - size_t len; int blksz, error, r, resid; swa = &ses->swcr_auth; @@ -1017,45 +1017,71 @@ swcr_chacha20_poly1305(const struct swcr_session *ses, struct cryptop *crp) /* Do encryption with MAC */ crypto_cursor_init(&cc_in, &crp->crp_buf); crypto_cursor_advance(&cc_in, crp->crp_payload_start); + inblk = crypto_cursor_segment(&cc_in, &inlen); if (CRYPTO_HAS_OUTPUT_BUFFER(crp)) { crypto_cursor_init(&cc_out, &crp->crp_obuf); crypto_cursor_advance(&cc_out, crp->crp_payload_output_start); } else cc_out = cc_in; - for (resid = crp->crp_payload_length; resid >= blksz; resid -= blksz) { - inblk = crypto_cursor_segment(&cc_in, &len); - if (len < blksz) { - crypto_cursor_copydata(&cc_in, blksz, blk); - inblk = blk; - } else - crypto_cursor_advance(&cc_in, blksz); - if (CRYPTO_OP_IS_ENCRYPT(crp->crp_op)) { - outblk = crypto_cursor_segment(&cc_out, &len); - if (len < blksz) + outblk = crypto_cursor_segment(&cc_out, &outlen); + + if (CRYPTO_OP_IS_ENCRYPT(crp->crp_op)) { + for (resid = crp->crp_payload_length; resid >= blksz; + resid -= todo) { + if (inlen < blksz) { + crypto_cursor_copydata(&cc_in, blksz, blk); + inblk = blk; + inlen = blksz; + } + + if (outlen < blksz) { outblk = blk; - exf->encrypt(ctx, inblk, outblk); - exf->update(ctx, outblk, blksz); - if (outblk == blk) + outlen = blksz; + } + + todo = rounddown2(MIN(resid, MIN(inlen, outlen)), + blksz); + + exf->encrypt_multi(ctx, inblk, outblk, todo); + exf->update(ctx, outblk, todo); + + if (inblk == blk) { + inblk = crypto_cursor_segment(&cc_in, &inlen); + } else { + crypto_cursor_advance(&cc_in, todo); + inlen -= todo; + inblk += todo; + if (inlen == 0) + inblk = crypto_cursor_segment(&cc_in, + &inlen); + } + + if (outblk == blk) { crypto_cursor_copyback(&cc_out, blksz, blk); - else - crypto_cursor_advance(&cc_out, blksz); - } else { - exf->update(ctx, inblk, blksz); + outblk = crypto_cursor_segment(&cc_out, &outlen); + } else { + crypto_cursor_advance(&cc_out, todo); + outlen -= todo; + outblk += todo; + if (outlen == 0) + outblk = crypto_cursor_segment(&cc_out, + &outlen); + } } - } - if (resid > 0) { - crypto_cursor_copydata(&cc_in, resid, blk); - if (CRYPTO_OP_IS_ENCRYPT(crp->crp_op)) { + if (resid > 0) { + crypto_cursor_copydata(&cc_in, resid, blk); exf->encrypt_last(ctx, blk, blk, resid); crypto_cursor_copyback(&cc_out, resid, blk); + exf->update(ctx, blk, resid); } - exf->update(ctx, blk, resid); - if (resid % POLY1305_BLOCK_LEN != 0) { - /* padding2 */ - memset(blk, 0, POLY1305_BLOCK_LEN); - exf->update(ctx, blk, POLY1305_BLOCK_LEN - - resid % POLY1305_BLOCK_LEN); - } + } else + crypto_apply(crp, crp->crp_payload_start, + crp->crp_payload_length, exf->update, ctx); + if (crp->crp_payload_length % POLY1305_BLOCK_LEN != 0) { + /* padding2 */ + memset(blk, 0, POLY1305_BLOCK_LEN); + exf->update(ctx, blk, POLY1305_BLOCK_LEN - + crp->crp_payload_length % POLY1305_BLOCK_LEN); } /* lengths */ @@ -1081,22 +1107,48 @@ swcr_chacha20_poly1305(const struct swcr_session *ses, struct cryptop *crp) /* tag matches, decrypt data */ crypto_cursor_init(&cc_in, &crp->crp_buf); crypto_cursor_advance(&cc_in, crp->crp_payload_start); + inblk = crypto_cursor_segment(&cc_in, &inlen); + for (resid = crp->crp_payload_length; resid > blksz; - resid -= blksz) { - inblk = crypto_cursor_segment(&cc_in, &len); - if (len < blksz) { + resid -= todo) { + if (inlen < blksz) { crypto_cursor_copydata(&cc_in, blksz, blk); inblk = blk; - } else - crypto_cursor_advance(&cc_in, blksz); - outblk = crypto_cursor_segment(&cc_out, &len); - if (len < blksz) + inlen = blksz; + } + if (outlen < blksz) { outblk = blk; - exf->decrypt(ctx, inblk, outblk); - if (outblk == blk) + outlen = blksz; + } + + todo = rounddown2(MIN(resid, MIN(inlen, outlen)), + blksz); + + exf->decrypt_multi(ctx, inblk, outblk, todo); + + if (inblk == blk) { + inblk = crypto_cursor_segment(&cc_in, &inlen); + } else { + crypto_cursor_advance(&cc_in, todo); + inlen -= todo; + inblk += todo; + if (inlen == 0) + inblk = crypto_cursor_segment(&cc_in, + &inlen); + } + + if (outblk == blk) { crypto_cursor_copyback(&cc_out, blksz, blk); - else - crypto_cursor_advance(&cc_out, blksz); + outblk = crypto_cursor_segment(&cc_out, + &outlen); + } else { + crypto_cursor_advance(&cc_out, todo); + outlen -= todo; + outblk += todo; + if (outlen == 0) + outblk = crypto_cursor_segment(&cc_out, + &outlen); + } } if (resid > 0) { crypto_cursor_copydata(&cc_in, resid, blk);