From nobody Wed Mar 13 21:08:29 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tw35T1ZHyz5DgGQ; Wed, 13 Mar 2024 21:08:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tw35T14Knz4k3D; Wed, 13 Mar 2024 21:08:29 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710364109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9ZdQOPBwyrqhb/VrJycxjKgwhX9XwbqQKknDK+mA5lM=; b=ganBytNqd9b+JVFixfVuEv4YGK863PvJUETn1f8P847kojZFHkaftnhGiidc8sftn1QfkB vaCF5rZI3EGSpvb+Vk+fH0qej2ZLAGB2S0vI7lgHExVOuM5Fka/ZWdWOvOXPH75X6VKa90 9Bu/HrAr8cSC30pNWZhUiCkdZJpapalf23zljNExfUbU/OUZHsbsCU19mgTAmBZC3pHFFs BsOq0zbqvASYkQBDTdXFWUnAUb/c5l7prYU55AJMHgsjBpk3+n61xqJ9ZSnUVU30QiX6lR 7ZR4pEjcZNDbhGt+ZSS3QClHbzQP7XkLz0fDBY+i+F5WzzYp87sfh2tSfRdR/g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710364109; a=rsa-sha256; cv=none; b=KaS4HzXjCKXQyW2cRyjW+o54gGnkn74UY9QA+dEXvGeSRJCvlMQFHmWiT2ppvzaXHEee6S gXfXDx6Vc4BWeuXxcr0o616gFqisC1jWlERpRW7tO99RTLOktPns4JPrMMR5QwC9NXhQAd C9zjpFeSU/rybOjWDqfNu6lkdgjtCRqqowGogSYVYNzR9Qni06jlm9bYAUqxREyeUAVkYF /fevzmrkpiea493flKuO+MAmDY6WcfqDth4ttET0JZM8KeFLwcTKR7WZFsQqgh3ynYDutQ 0D3ksfZl7jjqGh28yNEAFR1hbeA+IWj0DQ63xLFiQFriVYdys8Ejh+ddSSBrIg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710364109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9ZdQOPBwyrqhb/VrJycxjKgwhX9XwbqQKknDK+mA5lM=; b=qu83iq9JQ2UeDtdV/cb0Wlx6yf9Nw8kU869/XBRpPDKEqnhI/NrlX51GoLCc+ZTlr+oZtG lF5mMxbbkxMi5aO2RBa74030qqGqN6W8eGV5KHpzBY1f6EIQLuA1ykpPHqZ9asx2SbOJAp siNiofhjrGvpUfisiFwOMVEolFC5jx4yQcgren7Kwv7nvAUndjlZcMvYzLQZJ2+0p/kU4S 0GOAfgJQ/M8weZshx3+Nt3dD1in7KcawlLQXq9jN7ICVyV1POfNcaTOBtW/Z9Un6o2+Rkb jMJ7TZmOUPvOa1kUAH34vRnaKsVfPuOD8BlUvLyF66jib+0KwCZDjxm81+JAUw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Tw35T0gWwzyGn; Wed, 13 Mar 2024 21:08:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 42DL8Tu6057296; Wed, 13 Mar 2024 21:08:29 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 42DL8TbO057293; Wed, 13 Mar 2024 21:08:29 GMT (envelope-from git) Date: Wed, 13 Mar 2024 21:08:29 GMT Message-Id: <202403132108.42DL8TbO057293@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Richard Scheffenegger Subject: git: 85df11a1dec6 - main - ktls: deep copy tls_enable struct for in-kernel tcp consumers List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rscheff X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 85df11a1dec6eab9efbce9fd20712402a8e7ac7c Auto-Submitted: auto-generated The branch main has been updated by rscheff: URL: https://cgit.FreeBSD.org/src/commit/?id=85df11a1dec6eab9efbce9fd20712402a8e7ac7c commit 85df11a1dec6eab9efbce9fd20712402a8e7ac7c Author: Richard Scheffenegger AuthorDate: 2024-03-13 11:35:51 +0000 Commit: Richard Scheffenegger CommitDate: 2024-03-13 12:23:13 +0000 ktls: deep copy tls_enable struct for in-kernel tcp consumers Doing a deep copy of the keys early allows users of the tls_enable structure to assume kernel memory. This enables the socket options to be set by kernel threads. Reviewed By: #transport, tuexen, jhb, rrs Sponsored by: NetApp, Inc. X-NetApp-PR: #79 Differential Revision: https://reviews.freebsd.org/D44250 --- sys/kern/uipc_ktls.c | 96 ++++++++++++++++++++++++++++++++++++++++-------- sys/netinet/tcp_usrreq.c | 44 ++++------------------ sys/sys/ktls.h | 17 +++++---- 3 files changed, 97 insertions(+), 60 deletions(-) diff --git a/sys/kern/uipc_ktls.c b/sys/kern/uipc_ktls.c index deba6940bbee..df296090ec97 100644 --- a/sys/kern/uipc_ktls.c +++ b/sys/kern/uipc_ktls.c @@ -297,10 +297,86 @@ SYSCTL_COUNTER_U64(_kern_ipc_tls_toe, OID_AUTO, chacha20, CTLFLAG_RD, static MALLOC_DEFINE(M_KTLS, "ktls", "Kernel TLS"); +static void ktls_reclaim_thread(void *ctx); static void ktls_reset_receive_tag(void *context, int pending); static void ktls_reset_send_tag(void *context, int pending); static void ktls_work_thread(void *ctx); -static void ktls_reclaim_thread(void *ctx); + +int +ktls_copyin_tls_enable(struct sockopt *sopt, struct tls_enable *tls) +{ + struct tls_enable_v0 tls_v0; + int error; + uint8_t *cipher_key = NULL, *iv = NULL, *auth_key = NULL; + + if (sopt->sopt_valsize == sizeof(tls_v0)) { + error = sooptcopyin(sopt, &tls_v0, sizeof(tls_v0), sizeof(tls_v0)); + if (error != 0) + goto done; + memset(tls, 0, sizeof(*tls)); + tls->cipher_key = tls_v0.cipher_key; + tls->iv = tls_v0.iv; + tls->auth_key = tls_v0.auth_key; + tls->cipher_algorithm = tls_v0.cipher_algorithm; + tls->cipher_key_len = tls_v0.cipher_key_len; + tls->iv_len = tls_v0.iv_len; + tls->auth_algorithm = tls_v0.auth_algorithm; + tls->auth_key_len = tls_v0.auth_key_len; + tls->flags = tls_v0.flags; + tls->tls_vmajor = tls_v0.tls_vmajor; + tls->tls_vminor = tls_v0.tls_vminor; + } else + error = sooptcopyin(sopt, tls, sizeof(*tls), sizeof(*tls)); + + if (error != 0) + goto done; + + /* + * Now do a deep copy of the variable-length arrays in the struct, so that + * subsequent consumers of it can reliably assume kernel memory. This + * requires doing our own allocations, which we will free in the + * error paths so that our caller need only worry about outstanding + * allocations existing on successful return. + */ + cipher_key = malloc(tls->cipher_key_len, M_KTLS, M_WAITOK); + iv = malloc(tls->iv_len, M_KTLS, M_WAITOK); + auth_key = malloc(tls->auth_key_len, M_KTLS, M_WAITOK); + if (sopt->sopt_td != NULL) { + error = copyin(tls->cipher_key, cipher_key, tls->cipher_key_len); + if (error != 0) + goto done; + error = copyin(tls->iv, iv, tls->iv_len); + if (error != 0) + goto done; + error = copyin(tls->auth_key, auth_key, tls->auth_key_len); + if (error != 0) + goto done; + } else { + bcopy(tls->cipher_key, cipher_key, tls->cipher_key_len); + bcopy(tls->iv, iv, tls->iv_len); + bcopy(tls->auth_key, auth_key, tls->auth_key_len); + } + tls->cipher_key = cipher_key; + tls->iv = iv; + tls->auth_key = auth_key; + +done: + if (error != 0) { + zfree(cipher_key, M_KTLS); + zfree(iv, M_KTLS); + zfree(auth_key, M_KTLS); + } + + return (error); +} + +void +ktls_cleanup_tls_enable(struct tls_enable *tls) +{ + zfree(__DECONST(void *, tls->cipher_key), M_KTLS); + zfree(__DECONST(void *, tls->iv), M_KTLS); + zfree(__DECONST(void *, tls->auth_key), M_KTLS); +} static u_int ktls_get_cpu(struct socket *so) @@ -702,18 +778,12 @@ ktls_create_session(struct socket *so, struct tls_enable *en, tls->params.auth_key_len = en->auth_key_len; tls->params.auth_key = malloc(en->auth_key_len, M_KTLS, M_WAITOK); - error = copyin(en->auth_key, tls->params.auth_key, - en->auth_key_len); - if (error) - goto out; + bcopy(en->auth_key, tls->params.auth_key, en->auth_key_len); } tls->params.cipher_key_len = en->cipher_key_len; tls->params.cipher_key = malloc(en->cipher_key_len, M_KTLS, M_WAITOK); - error = copyin(en->cipher_key, tls->params.cipher_key, - en->cipher_key_len); - if (error) - goto out; + bcopy(en->cipher_key, tls->params.cipher_key, en->cipher_key_len); /* * This holds the implicit portion of the nonce for AEAD @@ -722,9 +792,7 @@ ktls_create_session(struct socket *so, struct tls_enable *en, */ if (en->iv_len != 0) { tls->params.iv_len = en->iv_len; - error = copyin(en->iv, tls->params.iv, en->iv_len); - if (error) - goto out; + bcopy(en->iv, tls->params.iv, en->iv_len); /* * For TLS 1.2 with GCM, generate an 8-byte nonce as a @@ -740,10 +808,6 @@ ktls_create_session(struct socket *so, struct tls_enable *en, *tlsp = tls; return (0); - -out: - ktls_free(tls); - return (error); } static struct ktls_session * diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c index a73d2a15c1d5..916fe33e8704 100644 --- a/sys/netinet/tcp_usrreq.c +++ b/sys/netinet/tcp_usrreq.c @@ -1914,37 +1914,6 @@ CTASSERT(TCP_CA_NAME_MAX <= TCP_LOG_ID_LEN); CTASSERT(TCP_LOG_REASON_LEN <= TCP_LOG_ID_LEN); #endif -#ifdef KERN_TLS -static int -copyin_tls_enable(struct sockopt *sopt, struct tls_enable *tls) -{ - struct tls_enable_v0 tls_v0; - int error; - - if (sopt->sopt_valsize == sizeof(tls_v0)) { - error = sooptcopyin(sopt, &tls_v0, sizeof(tls_v0), - sizeof(tls_v0)); - if (error) - return (error); - memset(tls, 0, sizeof(*tls)); - tls->cipher_key = tls_v0.cipher_key; - tls->iv = tls_v0.iv; - tls->auth_key = tls_v0.auth_key; - tls->cipher_algorithm = tls_v0.cipher_algorithm; - tls->cipher_key_len = tls_v0.cipher_key_len; - tls->iv_len = tls_v0.iv_len; - tls->auth_algorithm = tls_v0.auth_algorithm; - tls->auth_key_len = tls_v0.auth_key_len; - tls->flags = tls_v0.flags; - tls->tls_vmajor = tls_v0.tls_vmajor; - tls->tls_vminor = tls_v0.tls_vminor; - return (0); - } - - return (sooptcopyin(sopt, tls, sizeof(*tls), sizeof(*tls))); -} -#endif - extern struct cc_algo newreno_cc_algo; static int @@ -2292,15 +2261,16 @@ unlock_and_done: #ifdef KERN_TLS case TCP_TXTLS_ENABLE: INP_WUNLOCK(inp); - error = copyin_tls_enable(sopt, &tls); - if (error) + error = ktls_copyin_tls_enable(sopt, &tls); + if (error != 0) break; error = ktls_enable_tx(so, &tls); + ktls_cleanup_tls_enable(&tls); break; case TCP_TXTLS_MODE: INP_WUNLOCK(inp); error = sooptcopyin(sopt, &ui, sizeof(ui), sizeof(ui)); - if (error) + if (error != 0) return (error); INP_WLOCK_RECHECK(inp); @@ -2309,11 +2279,11 @@ unlock_and_done: break; case TCP_RXTLS_ENABLE: INP_WUNLOCK(inp); - error = sooptcopyin(sopt, &tls, sizeof(tls), - sizeof(tls)); - if (error) + error = ktls_copyin_tls_enable(sopt, &tls); + if (error != 0) break; error = ktls_enable_rx(so, &tls); + ktls_cleanup_tls_enable(&tls); break; #endif case TCP_MAXUNACKTIME: diff --git a/sys/sys/ktls.h b/sys/sys/ktls.h index 693864394ffe..9b3433f4b1fd 100644 --- a/sys/sys/ktls.h +++ b/sys/sys/ktls.h @@ -174,6 +174,7 @@ struct m_snd_tag; struct mbuf; struct sockbuf; struct socket; +struct sockopt; struct ktls_session { struct ktls_ocf_session *ocf_session; @@ -213,27 +214,29 @@ typedef enum { } ktls_mbuf_crypto_st_t; void ktls_check_rx(struct sockbuf *sb); -ktls_mbuf_crypto_st_t ktls_mbuf_crypto_state(struct mbuf *mb, int offset, int len); +void ktls_cleanup_tls_enable(struct tls_enable *tls); +int ktls_copyin_tls_enable(struct sockopt *sopt, struct tls_enable *tls); void ktls_disable_ifnet(void *arg); int ktls_enable_rx(struct socket *so, struct tls_enable *en); int ktls_enable_tx(struct socket *so, struct tls_enable *en); +void ktls_enqueue(struct mbuf *m, struct socket *so, int page_count); +void ktls_enqueue_to_free(struct mbuf *m); void ktls_destroy(struct ktls_session *tls); void ktls_frame(struct mbuf *m, struct ktls_session *tls, int *enqueue_cnt, uint8_t record_type); -bool ktls_permit_empty_frames(struct ktls_session *tls); -void ktls_seq(struct sockbuf *sb, struct mbuf *m); -void ktls_enqueue(struct mbuf *m, struct socket *so, int page_count); -void ktls_enqueue_to_free(struct mbuf *m); int ktls_get_rx_mode(struct socket *so, int *modep); -int ktls_set_tx_mode(struct socket *so, int mode); int ktls_get_tx_mode(struct socket *so, int *modep); int ktls_get_rx_sequence(struct inpcb *inp, uint32_t *tcpseq, uint64_t *tlsseq); void ktls_input_ifp_mismatch(struct sockbuf *sb, struct ifnet *ifp); -int ktls_output_eagain(struct inpcb *inp, struct ktls_session *tls); +ktls_mbuf_crypto_st_t ktls_mbuf_crypto_state(struct mbuf *mb, int offset, int len); #ifdef RATELIMIT int ktls_modify_txrtlmt(struct ktls_session *tls, uint64_t max_pacing_rate); #endif +int ktls_output_eagain(struct inpcb *inp, struct ktls_session *tls); bool ktls_pending_rx_info(struct sockbuf *sb, uint64_t *seqnop, size_t *residp); +bool ktls_permit_empty_frames(struct ktls_session *tls); +void ktls_seq(struct sockbuf *sb, struct mbuf *m); +int ktls_set_tx_mode(struct socket *so, int mode); static inline struct ktls_session * ktls_hold(struct ktls_session *tls)