From nobody Fri Dec 29 23:08:56 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T21K500Cvz55TDx; Fri, 29 Dec 2023 23:08:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T21K46GYnz4Gg1; Fri, 29 Dec 2023 23:08:56 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703891336; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=u3ReI1c0aV04UrEv+fk0I4XnTlMiAo99wQtjGCH7FnM=; b=dXgCCMsoDZwPQK4WKku+9x6/Zvmv1LxoHnOrtt4peODz8Me16D7toPWjzrtydO8qB2VCpJ zToPsiw0w7tOZc/40SdrkNzBn+lEY+d2f2qqf8Dwz3634Uj2LbG39IO8yviir7VqcrSjx1 hW6FVd9odiRvpp9C6KULe3rkQb5cRWhFKU0SHtDjNdvGuPw6S8QJnwIx8pJb1nKbfcSiPe xtWzyFHGXZpunYGzzlZ9XDphvATp/n9YOlqMPlqtWnh0u34XUhuaD4t5FCRFRq/Uzgwjyz fA8COhEy9np8egub2xZKHxXjP11IflV6J1tTu19fpjAGGZJc2J61mXstcaFaSQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703891336; a=rsa-sha256; cv=none; b=IPjRKR6I8sdqt//BC0OuoEzaQIn8Hl64ql9hIOp5SrY7S+YWw98Dpv3E3VvWBlGJ4DAuvS o7E81mSKcaXKMob+A6I3iuOfT9qKRRqswXGEXyhe/oUqW7qlFHfcePDLuPj4QWa3PCw1yU b97vMNLf+RpVgZ7e9M46ilVn1VKgiuQ2B0mp0rVWJDkz3+233WmAeglcExM+oWfYjeVGcU /xtc/OzeTbxiAUYdsi5KFQlF3rLWngMq8jRDCKHHJIc5gEFFebZ3NQVGkwFT1Zy5a/Nbr5 154Tln141Aa/Y5CQVYYPxy8BXzueGb+lD6KVwplNzg+mAxvfDyLk/ew/1/vuOQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703891336; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=u3ReI1c0aV04UrEv+fk0I4XnTlMiAo99wQtjGCH7FnM=; b=YlXmRDMAwuBrpJWyAVWE9mLG62U1Cid5ujTMIa7MhYUx4Sp9uUJp3XV1exfcK2TrUP64jx BsTFoeI2gHYsNH9+dxDm52czctYVfxNBIKkt5WA4ENx3xOowDefM//qK9DCiX410IoHfDG MowZYoz7PK5t2wVT/UfmPaWA8TiLAEobDXXIAyL05FgfokI4CgyIJS8PFO6qwkipdknyLU 0D9QK/V+aSTgc5qSWas18aar1+4MHLt6fsHAhTsLoPaIxEXIsoBKAas89euqckjUo7P7sM jDuAICT9akwpkWd2de1eYBobvgVySVz5/5zIAT1DZQDL+MypYWFQRhlU7h2eZQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4T21K459jmz9HK; Fri, 29 Dec 2023 23:08:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 3BTN8ujU075581; Fri, 29 Dec 2023 23:08:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 3BTN8ubS075578; Fri, 29 Dec 2023 23:08:56 GMT (envelope-from git) Date: Fri, 29 Dec 2023 23:08:56 GMT Message-Id: <202312292308.3BTN8ubS075578@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: 991f34a04a97 - stable/14 - armv8_crypto: Switch to using FPU_KERN_NOCTX List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 991f34a04a97e63747ae9dafc8004fdc36c421bb Auto-Submitted: auto-generated The branch stable/14 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=991f34a04a97e63747ae9dafc8004fdc36c421bb commit 991f34a04a97e63747ae9dafc8004fdc36c421bb Author: John Baldwin AuthorDate: 2023-08-28 23:22:33 +0000 Commit: John Baldwin CommitDate: 2023-12-29 21:49:42 +0000 armv8_crypto: Switch to using FPU_KERN_NOCTX Reviewed by: markj Differential Revision: https://reviews.freebsd.org/D41578 (cherry picked from commit 6485286f536f1afb36d6d39e75fceabd114b082b) --- sys/crypto/armv8/armv8_crypto.c | 85 +++++------------------------------------ 1 file changed, 9 insertions(+), 76 deletions(-) diff --git a/sys/crypto/armv8/armv8_crypto.c b/sys/crypto/armv8/armv8_crypto.c index a1a9d4dfcdcd..a8d88c31edc0 100644 --- a/sys/crypto/armv8/armv8_crypto.c +++ b/sys/crypto/armv8/armv8_crypto.c @@ -48,7 +48,6 @@ #include #include #include -#include #include #include #include @@ -69,22 +68,6 @@ struct armv8_crypto_softc { bool has_pmul; }; -static struct mtx *ctx_mtx; -static struct fpu_kern_ctx **ctx_vfp; - -#define AQUIRE_CTX(i, ctx) \ - do { \ - (i) = PCPU_GET(cpuid); \ - mtx_lock(&ctx_mtx[(i)]); \ - (ctx) = ctx_vfp[(i)]; \ - } while (0) -#define RELEASE_CTX(i, ctx) \ - do { \ - mtx_unlock(&ctx_mtx[(i)]); \ - (i) = -1; \ - (ctx) = NULL; \ - } while (0) - static int armv8_crypto_cipher_process(struct armv8_crypto_session *, struct cryptop *); @@ -134,7 +117,6 @@ armv8_crypto_attach(device_t dev) { struct armv8_crypto_softc *sc; uint64_t reg; - int i; sc = device_get_softc(dev); sc->dieing = 0; @@ -153,16 +135,6 @@ armv8_crypto_attach(device_t dev) rw_init(&sc->lock, "armv8crypto"); - ctx_mtx = malloc(sizeof(*ctx_mtx) * (mp_maxid + 1), M_ARMV8_CRYPTO, - M_WAITOK|M_ZERO); - ctx_vfp = malloc(sizeof(*ctx_vfp) * (mp_maxid + 1), M_ARMV8_CRYPTO, - M_WAITOK|M_ZERO); - - CPU_FOREACH(i) { - ctx_vfp[i] = fpu_kern_alloc_ctx(0); - mtx_init(&ctx_mtx[i], "armv8cryptoctx", NULL, MTX_DEF|MTX_NEW); - } - return (0); } @@ -170,7 +142,6 @@ static int armv8_crypto_detach(device_t dev) { struct armv8_crypto_softc *sc; - int i; sc = device_get_softc(dev); @@ -181,18 +152,6 @@ armv8_crypto_detach(device_t dev) rw_destroy(&sc->lock); - CPU_FOREACH(i) { - if (ctx_vfp[i] != NULL) { - mtx_destroy(&ctx_mtx[i]); - fpu_kern_free_ctx(ctx_vfp[i]); - } - ctx_vfp[i] = NULL; - } - free(ctx_mtx, M_ARMV8_CRYPTO); - ctx_mtx = NULL; - free(ctx_vfp, M_ARMV8_CRYPTO); - ctx_vfp = NULL; - return (0); } @@ -271,8 +230,6 @@ armv8_crypto_cipher_setup(struct armv8_crypto_session *ses, const struct crypto_session_params *csp, const uint8_t *key, int keylen) { __uint128_val_t H; - struct fpu_kern_ctx *ctx; - int kt, i; if (csp->csp_cipher_alg == CRYPTO_AES_XTS) keylen /= 2; @@ -286,12 +243,7 @@ armv8_crypto_cipher_setup(struct armv8_crypto_session *ses, return (EINVAL); } - kt = is_fpu_kern_thread(0); - if (!kt) { - AQUIRE_CTX(i, ctx); - fpu_kern_enter(curthread, ctx, - FPU_KERN_NORMAL | FPU_KERN_KTHR); - } + fpu_kern_enter(curthread, NULL, FPU_KERN_NORMAL | FPU_KERN_NOCTX); aes_v8_set_encrypt_key(key, keylen * 8, &ses->enc_schedule); @@ -312,10 +264,7 @@ armv8_crypto_cipher_setup(struct armv8_crypto_session *ses, gcm_init_v8(ses->Htable, H.u); } - if (!kt) { - fpu_kern_leave(curthread, ctx); - RELEASE_CTX(i, ctx); - } + fpu_kern_leave(curthread, NULL); return (0); } @@ -378,12 +327,10 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, { struct crypto_buffer_cursor fromc, toc; const struct crypto_session_params *csp; - struct fpu_kern_ctx *ctx; uint8_t *authbuf; uint8_t iv[AES_BLOCK_LEN], tag[GMAC_DIGEST_LEN]; - int authallocated, i; + int authallocated; int encflag; - int kt; int error; csp = crypto_get_params(crp->crp_session); @@ -391,7 +338,6 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, authallocated = 0; authbuf = NULL; - kt = 1; if (csp->csp_cipher_alg == CRYPTO_AES_NIST_GCM_16) { if (crp->crp_aad != NULL) @@ -399,10 +345,8 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, else authbuf = armv8_crypto_cipher_alloc(crp, crp->crp_aad_start, crp->crp_aad_length, &authallocated); - if (authbuf == NULL) { - error = ENOMEM; - goto out; - } + if (authbuf == NULL) + return (ENOMEM); } crypto_cursor_init(&fromc, &crp->crp_buf); crypto_cursor_advance(&fromc, crp->crp_payload_start); @@ -413,12 +357,7 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, crypto_cursor_copy(&fromc, &toc); } - kt = is_fpu_kern_thread(0); - if (!kt) { - AQUIRE_CTX(i, ctx); - fpu_kern_enter(curthread, ctx, - FPU_KERN_NORMAL | FPU_KERN_KTHR); - } + fpu_kern_enter(curthread, NULL, FPU_KERN_NORMAL | FPU_KERN_NOCTX); if (crp->crp_cipher_key != NULL) { armv8_crypto_cipher_setup(ses, csp, crp->crp_cipher_key, @@ -427,11 +366,12 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, crypto_read_iv(crp, iv); + error = 0; switch (csp->csp_cipher_alg) { case CRYPTO_AES_CBC: if ((crp->crp_payload_length % AES_BLOCK_LEN) != 0) { error = EINVAL; - goto out; + break; } if (encflag) armv8_aes_encrypt_cbc(&ses->enc_schedule, @@ -464,18 +404,11 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, error = armv8_aes_decrypt_gcm(&ses->enc_schedule, crp->crp_payload_length, &fromc, &toc, crp->crp_aad_length, authbuf, tag, iv, ses->Htable); - if (error != 0) - goto out; } break; } - error = 0; -out: - if (!kt) { - fpu_kern_leave(curthread, ctx); - RELEASE_CTX(i, ctx); - } + fpu_kern_leave(curthread, NULL); if (authallocated) zfree(authbuf, M_ARMV8_CRYPTO);