From nobody Tue Jan 13 17:00:48 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4drFr05JRWz6NQGQ for ; Tue, 13 Jan 2026 17:00:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4drFr041PNz3c6Y for ; Tue, 13 Jan 2026 17:00:48 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1768323648; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9jkmSriHSbCdglKMIGKpLq5gXfq7NuB4Lf7Tzsh0GLw=; b=fVkGkCYOh0gpe4B5ETsEd79HVgSl3eHdOT2fEzlMEz2LGoY472E49bZX5d5zB1h+BzxpQ3 ovJ4X59FVAk2I7Dj7ntzyfI7ArlXzEHo+Ho4sFL8ObTkprxfSx9ySJ32797HI4/EnDYxzp MZUTjnNXnSVWwGoJVUTv5a6o8kuMUf2pD0nz3XHkBw21fqPaOyqR+dLpII0UiuB1lsnW45 LC8eTif5cN97Nl1nzSkiyoczm5+7DSrfgYVdPdbt02LdxVg9ON8p45AN1lk1ijir6o7R8B yJiMHmHOu1ju/BLyXIYZ8IdPuja+BsaRStskYmLArw05IGtakyeDkROqla6QdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1768323648; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9jkmSriHSbCdglKMIGKpLq5gXfq7NuB4Lf7Tzsh0GLw=; b=ZLG6kw6iOzPygZog3Mt9f5kARWjSqE8Jf1NkZRPGDKh3p6MaDUj0iATwWZrYgPr5cYV5XV DMoQAt71mtzuziK91euXcUM5s7mNzdwknM2ulPSidJ8WlQf6oip9aq0z6yfjEiWMzJiS1Q afa7aOHjX2fvw8DGbrcok1Jag2I0ZBYTIin9gWdcsnTDwB6ggfisQdpmqqocsqPWOsTzbU nOG/BZ4C1ibnRivQZm5PBNnilsUDu+4Ptg8eyzipatJzMJn4tK7Btdw6iowkos2qnP/qla dTsEa0zPgkHFqyn4s+6SAoyQoRi13NR6dFHybdzbKBB3T6cFOP6VssuuFNUaVw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1768323648; a=rsa-sha256; cv=none; b=yfQQdT7W/d2Ma7+vUnvrOlqeo0ofTEMGL1t8b8vqXSHeAmvekmET78xoeduxDDhjFpi0FS qrnM7BkAjZxZkFvzPjPaFRgzypqmHVvrmAAOjq3TuaidXLDBdFVPBb3vwDbH8kyUwhWmIN 8CXe/7thSA+aOLouExH+4jN6kO5EYiUdphfPdJa7BpbVQlhaanKk7mXuU9cmMdQdcUuHyh UFpkCPEiclycEne4y4m/H1nnZx50RfRGMq89mTwFfOxCa/O8ddzu3oKwIbz9nZ8hcmhWKD /YfBvgeUmQAlY1NKS8jZawOn20JI6Xre+ABjZQWXAxAnuetjOXL2yvifiH8/DQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4drFr037y6z5Kj for ; Tue, 13 Jan 2026 17:00:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 35124 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 13 Jan 2026 17:00:48 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Sarah Walker From: Andrew Turner Subject: git: 591c7a08bf8a - main - arm64: Enable MOPS in userspace List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 591c7a08bf8addce4b047ef9c8033c33099e4688 Auto-Submitted: auto-generated Date: Tue, 13 Jan 2026 17:00:48 +0000 Message-Id: <69667a40.35124.634b8053@gitrepo.freebsd.org> The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=591c7a08bf8addce4b047ef9c8033c33099e4688 commit 591c7a08bf8addce4b047ef9c8033c33099e4688 Author: Sarah Walker AuthorDate: 2026-01-13 14:18:31 +0000 Commit: Andrew Turner CommitDate: 2026-01-13 15:28:03 +0000 arm64: Enable MOPS in userspace Detect presence of FEAT_MOPS, and enable instruction set and set HWCAP2 flag if present. Add handler for MOE exceptions. Reviewed by: andrew Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D54558 --- sys/arm64/arm64/identcpu.c | 8 +++++- sys/arm64/arm64/machdep.c | 35 +++++++++++++++++++++++++ sys/arm64/arm64/trap.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 106 insertions(+), 1 deletion(-) diff --git a/sys/arm64/arm64/identcpu.c b/sys/arm64/arm64/identcpu.c index 2d07420bcdb0..e37c9813582b 100644 --- a/sys/arm64/arm64/identcpu.c +++ b/sys/arm64/arm64/identcpu.c @@ -1094,6 +1094,11 @@ static const struct mrs_field_value id_aa64isar2_mops[] = { MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64isar2_mops_caps[] = { + MRS_HWCAP(2, HWCAP2_MOPS, ID_AA64ISAR2_MOPS_IMPL), + MRS_HWCAP_END +}; + static const struct mrs_field_value id_aa64isar2_apa3[] = { MRS_FIELD_VALUE(ID_AA64ISAR2_APA3_NONE, ""), MRS_FIELD_VALUE(ID_AA64ISAR2_APA3_PAC, "APA3 PAC"), @@ -1149,7 +1154,8 @@ static const struct mrs_field id_aa64isar2_fields[] = { MRS_FIELD(ID_AA64ISAR2, PAC_frac, false, MRS_LOWER, 0, id_aa64isar2_pac_frac), MRS_FIELD(ID_AA64ISAR2, BC, false, MRS_LOWER, 0, id_aa64isar2_bc), - MRS_FIELD(ID_AA64ISAR2, MOPS, false, MRS_LOWER, 0, id_aa64isar2_mops), + MRS_FIELD_HWCAP(ID_AA64ISAR2, MOPS, false, MRS_LOWER, MRS_USERSPACE, + id_aa64isar2_mops, id_aa64isar2_mops_caps), MRS_FIELD_HWCAP(ID_AA64ISAR2, APA3, false, MRS_LOWER, MRS_USERSPACE, id_aa64isar2_apa3, id_aa64isar2_apa3_caps), MRS_FIELD_HWCAP(ID_AA64ISAR2, GPA3, false, MRS_LOWER, MRS_USERSPACE, diff --git a/sys/arm64/arm64/machdep.c b/sys/arm64/arm64/machdep.c index 322bad273a08..6790f47a0f82 100644 --- a/sys/arm64/arm64/machdep.c +++ b/sys/arm64/arm64/machdep.c @@ -219,6 +219,41 @@ CPU_FEAT(feat_pan, "Privileged access never", pan_check, NULL, pan_enable, pan_disabled, CPU_FEAT_AFTER_DEV | CPU_FEAT_PER_CPU); +static cpu_feat_en +mops_check(const struct cpu_feat *feat __unused, u_int midr __unused) +{ + uint64_t id_aa64isar2; + + if (!get_kernel_reg(ID_AA64ISAR2_EL1, &id_aa64isar2)) + return (FEAT_ALWAYS_DISABLE); + if (ID_AA64ISAR2_MOPS_VAL(id_aa64isar2) == ID_AA64ISAR2_MOPS_NONE) + return (FEAT_ALWAYS_DISABLE); + + return (FEAT_DEFAULT_ENABLE); +} + +static bool +mops_enable(const struct cpu_feat *feat __unused, + cpu_feat_errata errata_status __unused, u_int *errata_list __unused, + u_int errata_count __unused) +{ + WRITE_SPECIALREG(sctlr_el1, READ_SPECIALREG(sctlr_el1) | SCTLR_MSCEn); + isb(); + + return (true); +} + +static void +mops_disabled(const struct cpu_feat *feat __unused) +{ + WRITE_SPECIALREG(sctlr_el1, READ_SPECIALREG(sctlr_el1) & ~SCTLR_MSCEn); + isb(); +} + +CPU_FEAT(feat_mops, "MOPS", + mops_check, NULL, mops_enable, mops_disabled, + CPU_FEAT_AFTER_DEV | CPU_FEAT_PER_CPU); + bool has_hyp(void) { diff --git a/sys/arm64/arm64/trap.c b/sys/arm64/arm64/trap.c index 75c9b5f87892..3de56187657c 100644 --- a/sys/arm64/arm64/trap.c +++ b/sys/arm64/arm64/trap.c @@ -597,6 +597,66 @@ do_el1h_sync(struct thread *td, struct trapframe *frame) } } +static void +handle_moe(struct thread *td, struct trapframe *frame, uint64_t esr) +{ + uint64_t src; + uint64_t dest; + uint64_t size; + int src_reg; + int dest_reg; + int size_reg; + int format_option; + + format_option = esr & ISS_MOE_FORMAT_OPTION_MASK; + dest_reg = (esr & ISS_MOE_DESTREG_MASK) >> ISS_MOE_DESTREG_SHIFT; + size_reg = (esr & ISS_MOE_SIZEREG_MASK) >> ISS_MOE_SIZEREG_SHIFT; + dest = frame->tf_x[dest_reg]; + size = frame->tf_x[size_reg]; + + /* + * Put the registers back in the original format suitable for a + * prologue instruction, using the generic return routine from the + * Arm ARM (DDI 0487I.a) rules CNTMJ and MWFQH. + */ + if (esr & ISS_MOE_MEMINST) { + /* SET* instruction */ + if (format_option == ISS_MOE_FORMAT_OPTION_A || + format_option == ISS_MOE_FORMAT_OPTION_A2) { + /* Format is from Option A; forward set */ + frame->tf_x[dest_reg] = dest + size; + frame->tf_x[size_reg] = -size; + } + } else { + /* CPY* instruction */ + src_reg = (esr & ISS_MOE_SRCREG_MASK) >> ISS_MOE_SRCREG_SHIFT; + src = frame->tf_x[src_reg]; + + if (format_option == ISS_MOE_FORMAT_OPTION_B || + format_option == ISS_MOE_FORMAT_OPTION_B2) { + /* Format is from Option B */ + if (frame->tf_spsr & PSR_N) { + /* Backward copy */ + frame->tf_x[dest_reg] = dest - size; + frame->tf_x[src_reg] = src + size; + } + } else { + /* Format is from Option A */ + if (frame->tf_x[size_reg] & (1UL << 63)) { + /* Forward copy */ + frame->tf_x[dest_reg] = dest + size; + frame->tf_x[src_reg] = src + size; + frame->tf_x[size_reg] = -size; + } + } + } + + if (esr & ISS_MOE_FROM_EPILOGUE) + frame->tf_elr -= 8; + else + frame->tf_elr -= 4; +} + void do_el0_sync(struct thread *td, struct trapframe *frame) { @@ -738,6 +798,10 @@ do_el0_sync(struct thread *td, struct trapframe *frame) exception); userret(td, frame); break; + case EXCP_MOE: + handle_moe(td, frame, esr); + userret(td, frame); + break; default: call_trapsignal(td, SIGBUS, BUS_OBJERR, (void *)frame->tf_elr, exception);