From nobody Tue Nov 23 23:12:47 2021 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E213318A1795; Tue, 23 Nov 2021 23:12:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HzKg51KrSz3vVt; Tue, 23 Nov 2021 23:12:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 915881F2B7; Tue, 23 Nov 2021 23:12:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1ANNClYt037537; Tue, 23 Nov 2021 23:12:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1ANNClBV037536; Tue, 23 Nov 2021 23:12:47 GMT (envelope-from git) Date: Tue, 23 Nov 2021 23:12:47 GMT Message-Id: <202111232312.1ANNClBV037536@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: 98641c00a3ae - stable/13 - Add Chacha20-Poly1305 support in the OCF backend for KTLS. List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 98641c00a3aef5d82da653b9a50c734fb4b08d87 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1637709170; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=rjloV9SmKSdgGX41x/RrknxyqaXxEvrf4i/H/zm0b4Y=; b=ozwxRDgU0ITcknB8CsMQzCsD2w8hFhV0OTqAUeC2CHVsRYaQQCB7hU+Q/oKtHFL67wYqsi T9dJsUyEJJ8qYPjbtTJaHBdbYz0HQ6UerhKCFpMsteI1Jf/e1T89icCFxxDp2Z8UZLb3rs Lj0crt5/nrBhvR17jNPcCByMJpS0MoG7StpFXBQmUPv/VdA6+jQIQXfuXLTgHYwFYWpfVC 7pbAYrG1yZ9Qz8VqG4VEnLCJLrsMhCuhPB1hhvaHw+sN7k4TmqxlX+z8otyLH7gqUQetd1 VCqHLiM+OCO1LVN3vZS2JJ0HzYGDeVphlBF6uLM1k0UkvOmpXMKAOn72o1pA8g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1637709170; a=rsa-sha256; cv=none; b=Ez2Fy5i3tzx/gdyGwkAhxhdDwAJD6Jvc9lTjPtRadKka9xpQ9kU2udolADZCTeQgQv+7mr 2f2j/BwSDevCQ2jb2+3s2qSTQIcj5mSx0IcmLrsFJF3X/+gKkgOlBWMV0CIhdzH/7RND9Z hXgMhkuUbCUt0Gj4EqZDS8QVYdxqJdAqP6fBTbNtym9TTFDyBN+0IiiVBOg2SgJjXVli3M +qTmMnS9d56MXp7rywtaa4oyE4hJ6L01U0ddTKB3bcVs/AeB7ytNHK5/I/Dl8RtEgL3O3L AGWTezIrIsroZPQhAqSNKxfnDs64TXFci1/gvgYEApbbjeuWkyA0rt+nL+DidA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=98641c00a3aef5d82da653b9a50c734fb4b08d87 commit 98641c00a3aef5d82da653b9a50c734fb4b08d87 Author: John Baldwin AuthorDate: 2021-02-18 17:24:26 +0000 Commit: John Baldwin CommitDate: 2021-11-23 23:11:44 +0000 Add Chacha20-Poly1305 support in the OCF backend for KTLS. This supports Chacha20-Poly1305 for both send and receive for TLS 1.2 and for send in TLS 1.3. Reviewed by: gallatin Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D27841 (cherry picked from commit 4dd6800e22b08fa1f756115600e9436818abb168) --- sys/opencrypto/ktls_ocf.c | 116 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 95 insertions(+), 21 deletions(-) diff --git a/sys/opencrypto/ktls_ocf.c b/sys/opencrypto/ktls_ocf.c index 7f9ece99ccb1..1d5dce83b376 100644 --- a/sys/opencrypto/ktls_ocf.c +++ b/sys/opencrypto/ktls_ocf.c @@ -87,11 +87,21 @@ SYSCTL_COUNTER_U64(_kern_ipc_tls_stats_ocf, OID_AUTO, tls12_gcm_crypts, CTLFLAG_RD, &ocf_tls12_gcm_crypts, "Total number of OCF TLS 1.2 GCM encryption operations"); +static COUNTER_U64_DEFINE_EARLY(ocf_tls12_chacha20_crypts); +SYSCTL_COUNTER_U64(_kern_ipc_tls_stats_ocf, OID_AUTO, tls12_chacha20_crypts, + CTLFLAG_RD, &ocf_tls12_chacha20_crypts, + "Total number of OCF TLS 1.2 Chacha20-Poly1305 encryption operations"); + static COUNTER_U64_DEFINE_EARLY(ocf_tls13_gcm_crypts); SYSCTL_COUNTER_U64(_kern_ipc_tls_stats_ocf, OID_AUTO, tls13_gcm_crypts, CTLFLAG_RD, &ocf_tls13_gcm_crypts, "Total number of OCF TLS 1.3 GCM encryption operations"); +static COUNTER_U64_DEFINE_EARLY(ocf_tls13_chacha20_crypts); +SYSCTL_COUNTER_U64(_kern_ipc_tls_stats_ocf, OID_AUTO, tls13_chacha20_crypts, + CTLFLAG_RD, &ocf_tls13_chacha20_crypts, + "Total number of OCF TLS 1.3 Chacha20-Poly1305 encryption operations"); + static COUNTER_U64_DEFINE_EARLY(ocf_inplace); SYSCTL_COUNTER_U64(_kern_ipc_tls_stats_ocf, OID_AUTO, inplace, CTLFLAG_RD, &ocf_inplace, @@ -315,7 +325,7 @@ ktls_ocf_tls_cbc_encrypt(struct ktls_session *tls, } static int -ktls_ocf_tls12_gcm_encrypt(struct ktls_session *tls, +ktls_ocf_tls12_aead_encrypt(struct ktls_session *tls, const struct tls_record_layer *hdr, uint8_t *trailer, struct iovec *iniov, struct iovec *outiov, int iovcnt, uint64_t seqno, uint8_t record_type __unused) @@ -346,12 +356,26 @@ ktls_ocf_tls12_gcm_encrypt(struct ktls_session *tls, crypto_initreq(&crp, os->sid); /* Setup the IV. */ - memcpy(crp.crp_iv, tls->params.iv, TLS_AEAD_GCM_LEN); - memcpy(crp.crp_iv + TLS_AEAD_GCM_LEN, hdr + 1, sizeof(uint64_t)); + if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16) { + memcpy(crp.crp_iv, tls->params.iv, TLS_AEAD_GCM_LEN); + memcpy(crp.crp_iv + TLS_AEAD_GCM_LEN, hdr + 1, + sizeof(uint64_t)); + } else { + /* + * Chacha20-Poly1305 constructs the IV for TLS 1.2 + * identically to constructing the IV for AEAD in TLS + * 1.3. + */ + memcpy(crp.crp_iv, tls->params.iv, tls->params.iv_len); + *(uint64_t *)(crp.crp_iv + 4) ^= htobe64(seqno); + } /* Setup the AAD. */ - tls_comp_len = ntohs(hdr->tls_length) - - (AES_GMAC_HASH_LEN + sizeof(uint64_t)); + if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16) + tls_comp_len = ntohs(hdr->tls_length) - + (AES_GMAC_HASH_LEN + sizeof(uint64_t)); + else + tls_comp_len = ntohs(hdr->tls_length) - POLY1305_HASH_LEN; ad.seq = htobe64(seqno); ad.type = hdr->tls_type; ad.tls_vmajor = hdr->tls_vmajor; @@ -391,7 +415,10 @@ ktls_ocf_tls12_gcm_encrypt(struct ktls_session *tls, if (!inplace) crypto_use_output_uio(&crp, &out_uio); - counter_u64_add(ocf_tls12_gcm_crypts, 1); + if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16) + counter_u64_add(ocf_tls12_gcm_crypts, 1); + else + counter_u64_add(ocf_tls12_chacha20_crypts, 1); if (inplace) counter_u64_add(ocf_inplace, 1); else @@ -403,7 +430,7 @@ ktls_ocf_tls12_gcm_encrypt(struct ktls_session *tls, } static int -ktls_ocf_tls12_gcm_decrypt(struct ktls_session *tls, +ktls_ocf_tls12_aead_decrypt(struct ktls_session *tls, const struct tls_record_layer *hdr, struct mbuf *m, uint64_t seqno, int *trailer_len) { @@ -422,12 +449,26 @@ ktls_ocf_tls12_gcm_decrypt(struct ktls_session *tls, crypto_initreq(&crp, os->sid); /* Setup the IV. */ - memcpy(crp.crp_iv, tls->params.iv, TLS_AEAD_GCM_LEN); - memcpy(crp.crp_iv + TLS_AEAD_GCM_LEN, hdr + 1, sizeof(uint64_t)); + if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16) { + memcpy(crp.crp_iv, tls->params.iv, TLS_AEAD_GCM_LEN); + memcpy(crp.crp_iv + TLS_AEAD_GCM_LEN, hdr + 1, + sizeof(uint64_t)); + } else { + /* + * Chacha20-Poly1305 constructs the IV for TLS 1.2 + * identically to constructing the IV for AEAD in TLS + * 1.3. + */ + memcpy(crp.crp_iv, tls->params.iv, tls->params.iv_len); + *(uint64_t *)(crp.crp_iv + 4) ^= htobe64(seqno); + } /* Setup the AAD. */ - tls_comp_len = ntohs(hdr->tls_length) - - (AES_GMAC_HASH_LEN + sizeof(uint64_t)); + if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16) + tls_comp_len = ntohs(hdr->tls_length) - + (AES_GMAC_HASH_LEN + sizeof(uint64_t)); + else + tls_comp_len = ntohs(hdr->tls_length) - POLY1305_HASH_LEN; ad.seq = htobe64(seqno); ad.type = hdr->tls_type; ad.tls_vmajor = hdr->tls_vmajor; @@ -444,7 +485,10 @@ ktls_ocf_tls12_gcm_decrypt(struct ktls_session *tls, crp.crp_flags = CRYPTO_F_CBIMM | CRYPTO_F_IV_SEPARATE; crypto_use_mbuf(&crp, m); - counter_u64_add(ocf_tls12_gcm_crypts, 1); + if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16) + counter_u64_add(ocf_tls12_gcm_crypts, 1); + else + counter_u64_add(ocf_tls12_chacha20_crypts, 1); error = ktls_ocf_dispatch(os, &crp); crypto_destroyreq(&crp); @@ -453,7 +497,7 @@ ktls_ocf_tls12_gcm_decrypt(struct ktls_session *tls, } static int -ktls_ocf_tls13_gcm_encrypt(struct ktls_session *tls, +ktls_ocf_tls13_aead_encrypt(struct ktls_session *tls, const struct tls_record_layer *hdr, uint8_t *trailer, struct iovec *iniov, struct iovec *outiov, int iovcnt, uint64_t seqno, uint8_t record_type) { @@ -503,11 +547,11 @@ ktls_ocf_tls13_gcm_encrypt(struct ktls_session *tls, */ memcpy(iov, iniov, iovcnt * sizeof(*iov)); iov[iovcnt].iov_base = trailer; - iov[iovcnt].iov_len = AES_GMAC_HASH_LEN + 1; + iov[iovcnt].iov_len = tls->params.tls_tlen; uio.uio_iov = iov; uio.uio_iovcnt = iovcnt + 1; uio.uio_offset = 0; - uio.uio_resid = crp.crp_payload_length + AES_GMAC_HASH_LEN; + uio.uio_resid = crp.crp_payload_length + tls->params.tls_tlen - 1; uio.uio_segflg = UIO_SYSSPACE; uio.uio_td = curthread; crypto_use_uio(&crp, &uio); @@ -521,7 +565,7 @@ ktls_ocf_tls13_gcm_encrypt(struct ktls_session *tls, out_uio.uio_iovcnt = iovcnt + 1; out_uio.uio_offset = 0; out_uio.uio_resid = crp.crp_payload_length + - AES_GMAC_HASH_LEN; + tls->params.tls_tlen - 1; out_uio.uio_segflg = UIO_SYSSPACE; out_uio.uio_td = curthread; crypto_use_output_uio(&crp, &out_uio); @@ -532,7 +576,10 @@ ktls_ocf_tls13_gcm_encrypt(struct ktls_session *tls, memcpy(crp.crp_iv, nonce, sizeof(nonce)); - counter_u64_add(ocf_tls13_gcm_crypts, 1); + if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16) + counter_u64_add(ocf_tls13_gcm_crypts, 1); + else + counter_u64_add(ocf_tls13_chacha20_crypts, 1); if (inplace) counter_u64_add(ocf_inplace, 1); else @@ -640,6 +687,32 @@ ktls_ocf_try(struct socket *so, struct ktls_session *tls, int direction) mac_csp.csp_auth_key = tls->params.auth_key; mac_csp.csp_auth_klen = tls->params.auth_key_len; break; + case CRYPTO_CHACHA20_POLY1305: + switch (tls->params.cipher_key_len) { + case 256 / 8: + break; + default: + return (EINVAL); + } + + /* Only TLS 1.2 and 1.3 are supported. */ + if (tls->params.tls_vmajor != TLS_MAJOR_VER_ONE || + tls->params.tls_vminor < TLS_MINOR_VER_TWO || + tls->params.tls_vminor > TLS_MINOR_VER_THREE) + return (EPROTONOSUPPORT); + + /* TLS 1.3 is not yet supported for receive. */ + if (direction == KTLS_RX && + tls->params.tls_vminor == TLS_MINOR_VER_THREE) + return (EPROTONOSUPPORT); + + csp.csp_flags |= CSP_F_SEPARATE_OUTPUT | CSP_F_SEPARATE_AAD; + csp.csp_mode = CSP_MODE_AEAD; + csp.csp_cipher_alg = CRYPTO_CHACHA20_POLY1305; + csp.csp_cipher_key = tls->params.cipher_key; + csp.csp_cipher_klen = tls->params.cipher_key_len; + csp.csp_ivlen = CHACHA20_POLY1305_IV_LEN; + break; default: return (EPROTONOSUPPORT); } @@ -668,14 +741,15 @@ ktls_ocf_try(struct socket *so, struct ktls_session *tls, int direction) mtx_init(&os->lock, "ktls_ocf", NULL, MTX_DEF); tls->cipher = os; - if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16) { + if (tls->params.cipher_algorithm == CRYPTO_AES_NIST_GCM_16 || + tls->params.cipher_algorithm == CRYPTO_CHACHA20_POLY1305) { if (direction == KTLS_TX) { if (tls->params.tls_vminor == TLS_MINOR_VER_THREE) - tls->sw_encrypt = ktls_ocf_tls13_gcm_encrypt; + tls->sw_encrypt = ktls_ocf_tls13_aead_encrypt; else - tls->sw_encrypt = ktls_ocf_tls12_gcm_encrypt; + tls->sw_encrypt = ktls_ocf_tls12_aead_encrypt; } else { - tls->sw_decrypt = ktls_ocf_tls12_gcm_decrypt; + tls->sw_decrypt = ktls_ocf_tls12_aead_decrypt; } } else { tls->sw_encrypt = ktls_ocf_tls_cbc_encrypt;