From nobody Fri Dec 29 23:08:55 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T21K40Cmxz55TKY; Fri, 29 Dec 2023 23:08:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T21K352Jhz4Gfl; Fri, 29 Dec 2023 23:08:55 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703891335; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=W4D3MIaENswMHFg8RUVJaUToD+IQ2w2Mi1VnfDGVJpE=; b=ZrF9wfr90FHk2J1pkNX/iz64OjytSmeP/6TgWpdzrDhNjLje8qD+Rd68ZJx8Apj+UNpHRR ZHtF23ZLDI4fsEc78okwRP00UKthAO+7Ko6bqxoGal842mV9sjrl5Dx6wtWL93ratPG9/1 j91CqzYj7rXjU1+oKlM3RWJ6ngVi3ALQk3ssVQQZTZAMQs2xVTKvhxiOkOV7JZQgy9GidC WbsohciKwtHXGEv9uzmCLxVvMmP3wiD5dTGNkXBMGY5wboREpMhcn0JUQnY8l8HAXTsee7 bDY4FNB7qux7l/6uQV5MVJVfjwB+tECJntqu4DfANoEE0PEVhTmoULMcmT+I5A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703891335; a=rsa-sha256; cv=none; b=rDMHfa5znGBN640JBWIDeFDVhaUKTaIr4esVxlAMLP/TznB2+TSVa/nd6gz9zRzo+cO9Iw bHFI0z8dyTOp/WSGHZrUgMau8fhGHKk7umxVOkDjmS02mRFa2qkIit8CwRAjTHMlZJAUWs UsES39glxZLoBnakPJozHTd6vWoyThoVLSrzNCBdYnowRkfovFeJs6fGkMh/vG3Ti6cyIC PioBZ/EdAoA9b56UALddG5HA9JebzwzE5S5xBXhOYW25U0jsb6GfkyupDBg+ZxLRFLzoc7 BPA/00QAjSHBQT0F6PAEK1My3HccJc6cJwOBhpNCjuiCXsNjh89Zp9JMEGvDRQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703891335; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=W4D3MIaENswMHFg8RUVJaUToD+IQ2w2Mi1VnfDGVJpE=; b=La02rVFSj2doI3Y4LuqeU0DqEt2b5CiJuvrJa7bRwUuMAzt4P1uJmc9t7QM06o0H6HHJPE BFy3awnT/183edq+nQZrmJzVA8hQzHJn2kd88LqEEIx68w0p61O5hnj1yIOBMEvwjh0g2m gf4c+OG1UZVVi4jP8HmPxpDRtGvxpm5jV7txs7TXt4z9XnG9sPabPsfFePoj6eNuTTSGDg vqhq0T4kfPdGr/8JLv7bxEz1g2SJtzpH0rzjXXXphynvmE52Z1VzWvec5YopxkuZQLJyqm bpU1fRHO0t5WNoMoiRPYXWEBuSCb0CPI/5phU9s0/dA11vMd9AJVk9UVDtgKBw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4T21K348W5zBDK; Fri, 29 Dec 2023 23:08:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 3BTN8tdS075539; Fri, 29 Dec 2023 23:08:55 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 3BTN8tcx075536; Fri, 29 Dec 2023 23:08:55 GMT (envelope-from git) Date: Fri, 29 Dec 2023 23:08:55 GMT Message-Id: <202312292308.3BTN8tcx075536@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: 276666497ed1 - stable/14 - aesni: Push FPU sections down further List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 276666497ed1072ff663cb174eb44fe333be9ee3 Auto-Submitted: auto-generated The branch stable/14 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=276666497ed1072ff663cb174eb44fe333be9ee3 commit 276666497ed1072ff663cb174eb44fe333be9ee3 Author: Mark Johnston AuthorDate: 2023-08-29 01:26:53 +0000 Commit: John Baldwin CommitDate: 2023-12-29 21:49:42 +0000 aesni: Push FPU sections down further After commit 937b4473be21 aesni_cipher_crypt() and aesni_cipher_mac() execute in a FPU_KERN_NOCTX section, which means that they must run with preemption disabled. These functions handle discontiguous I/O buffers by allocating a contiguous buffer and copying as necessary, but this allocation cannot happen with preemption disabled. Fix the problem by pushing the FPU section down into aesni_cipher_crypt() and aesni_cipher_mac(). In particular, encrypt-then-auth transforms need not be handled with a single FPU section. Reported by: syzbot+78258dbb02eb92157357@syzkaller.appspotmail.com Discussed with: jhb Fixes: 937b4473be21 ("aesni: Switch to using FPU_KERN_NOCTX.") (cherry picked from commit 6b635c74fd4135eaae68970bfc5aad9ae905fec7) --- sys/crypto/aesni/aesni.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/sys/crypto/aesni/aesni.c b/sys/crypto/aesni/aesni.c index 6d83743dece9..5d5afd8aee41 100644 --- a/sys/crypto/aesni/aesni.c +++ b/sys/crypto/aesni/aesni.c @@ -594,8 +594,6 @@ aesni_cipher_process(struct aesni_session *ses, struct cryptop *crp) break; } - fpu_kern_enter(curthread, NULL, FPU_KERN_NORMAL | FPU_KERN_NOCTX); - /* Do work */ if (csp->csp_mode == CSP_MODE_ETA) { if (CRYPTO_OP_IS_ENCRYPT(crp->crp_op)) { @@ -612,7 +610,6 @@ aesni_cipher_process(struct aesni_session *ses, struct cryptop *crp) else error = aesni_cipher_crypt(ses, crp, csp); - fpu_kern_leave(curthread, NULL); return (error); } @@ -677,6 +674,8 @@ aesni_cipher_crypt(struct aesni_session *ses, struct cryptop *crp, outcopy = allocated; } + fpu_kern_enter(curthread, NULL, FPU_KERN_NORMAL | FPU_KERN_NOCTX); + error = 0; encflag = CRYPTO_OP_IS_ENCRYPT(crp->crp_op); if (crp->crp_cipher_key != NULL) @@ -749,6 +748,9 @@ aesni_cipher_crypt(struct aesni_session *ses, struct cryptop *crp, } break; } + + fpu_kern_leave(curthread, NULL); + if (outcopy && error == 0) crypto_copyback(crp, CRYPTO_HAS_OUTPUT_BUFFER(crp) ? crp->crp_payload_output_start : crp->crp_payload_start, @@ -784,6 +786,8 @@ aesni_cipher_mac(struct aesni_session *ses, struct cryptop *crp, key = csp->csp_auth_key; keylen = csp->csp_auth_klen; + fpu_kern_enter(curthread, NULL, FPU_KERN_NORMAL | FPU_KERN_NOCTX); + if (ses->hmac) { uint8_t hmac_key[SHA1_BLOCK_LEN] __aligned(16); @@ -849,6 +853,8 @@ aesni_cipher_mac(struct aesni_session *ses, struct cryptop *crp, ses->hash_finalize(res, &sctx); } + fpu_kern_leave(curthread, NULL); + if (crp->crp_op & CRYPTO_OP_VERIFY_DIGEST) { uint32_t res2[SHA2_256_HASH_LEN / sizeof(uint32_t)];