git: 2272fac218a8 - main - hwpmc: Add support for AMD-only IRPERF in hwpmc PERF class
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Tue, 13 Oct 2026 13:22:01 UTC
The branch main has been updated by mhorne:
URL: https://cgit.FreeBSD.org/src/commit/?id=2272fac218a88a6f25e696ec54ece2eedec39a38
commit 2272fac218a88a6f25e696ec54ece2eedec39a38
Author: Anderson Nascimento <anascime@amd.com>
AuthorDate: 2026-10-13 13:10:28 +0000
Commit: Mitchell Horne <mhorne@FreeBSD.org>
CommitDate: 2026-10-13 13:21:45 +0000
hwpmc: Add support for AMD-only IRPERF in hwpmc PERF class
The IRPERF MSR counter (0xc00000e9) exposes an instructions-retired count
when read. Its support is advertised through CPUID leaf 0x80000008 EBX bit 1.
If this bit is set, the CPU supports the IRPERF MSR. For it to be used
by the system, bit 30 of the HWCR MSR (0xC0010015) needs to be enabled.
This patch checks for availability of the feature and enables the bit. If
the feature is available, the hwpmc PERF class detects its presence and
registers a third PERF MSR counter that allows hwpmc users to measure
instructions retired using IRPERF. This is AMD-only, as Intel does not
support IRPERF.
Due to erratum #1054 affecting AMD Zen 1 before model 30h, IRPERF is only
enabled for AMD Zen 1 model 30h onwards and later microarchitectures. This
commit also implements a validation step to make sure the IRPERF counter
is actually incrementing; otherwise, the feature is not enabled. This
guarantees it is working as expected before being made available to the
system and users.
To support this, the perf_pmcdesc[] structure changed from being
statically initialized with 2 entries (MPERF/APERF) to being populated
dynamically. If both MPERF/APERF and IRPERF are supported, the structure
will contain three entries: MPERF, APERF, and IRPERF. Each entry now also
carries its own MSR address and PMC event, so perf_read_pmc() reads the
MSR to use from the matching perf_pmcdesc[] entry at runtime instead of
switching over a fixed set of row indices.
Definitions for the features available on the HWCR MSR did not previously
exist. This commit adds macros for those features, including
HWCR_IRPERFEN. The MSR definition for MSR_IRPERF was also missing and has
now been added.
This commit also updates the manual page accordingly.
Signed-off-by: Anderson Nascimento <anascime@amd.com>
Reviewed by: mhorne, adrian
Sponsored by: AMD
Differential Revision: https://reviews.freebsd.org/D59554
---
lib/libpmc/pmc.perf.3 | 79 +++++++++++++++++++++++----
sys/amd64/amd64/initcpu.c | 31 +++++++++++
sys/amd64/include/pmc_mdep.h | 2 +-
sys/dev/hwpmc/hwpmc_amd.c | 15 +++++-
sys/dev/hwpmc/hwpmc_perf.c | 124 +++++++++++++++++++++++--------------------
sys/dev/hwpmc/hwpmc_perf.h | 14 ++---
sys/dev/hwpmc/pmc_events.h | 7 +--
sys/x86/include/specialreg.h | 23 ++++++++
8 files changed, 215 insertions(+), 80 deletions(-)
diff --git a/lib/libpmc/pmc.perf.3 b/lib/libpmc/pmc.perf.3
index 2404abd53555..4595829eb362 100644
--- a/lib/libpmc/pmc.perf.3
+++ b/lib/libpmc/pmc.perf.3
@@ -3,12 +3,12 @@
.\"
.\" SPDX-License-Identifier: BSD-2-Clause
.\"
-.Dd August 5, 2026
+.Dd October 13, 2026
.Dt PMC.PERF 3
.Os
.Sh NAME
.Nm pmc.perf
-.Nd measurements using the MPERF/APERF effective frequency counters
+.Nd measurements using the MPERF/APERF/IRPERF frequency counters
.Sh LIBRARY
.Lb libpmc
.Sh SYNOPSIS
@@ -38,12 +38,24 @@ to
scaled by the processor's nominal frequency, yields the
average effective frequency.
.Pp
+AMD processors additionally provide a third fixed-counter machine
+specific register,
+.Va IRPERF
+.Pq Dq Instructions Retired Performance Count ,
+which increments by one for each instruction retired by the core.
+Unlike
+.Va MPERF
+and
+.Va APERF ,
+.Va IRPERF
+is an AMD-only facility and is not available on Intel processors.
+.Pp
The
.Dv PMC_CLASS_PERF
-class exposes these two MSRs as read-only 64-bit counters that may
+class exposes these MSRs as read-only 64-bit counters that may
only be allocated in system-wide counting mode
.Pq Dv PMC_MODE_SC.
-Both counters expose the unmodified contents of the underlying MSRs.
+All counters expose the unmodified contents of the underlying MSRs.
.Pp
It reflects the free-running count of the underlying MSR since the
processor was last reset, and continues to accumulate across
@@ -70,19 +82,29 @@ By default AMD/Intel PERF enables the read flag.
PERF events do not support further event qualifiers.
.Ss Event Specifiers
The following event names are supported:
-.Bl -tag -width "perf-mperf / mperf"
+.Bl -tag -width "perf-irperf / irperf"
.It Cm perf-mperf , Cm mperf
The Max Performance Frequency Clock Count
MSR of the core the counter is bound to.
.It Cm perf-aperf , Cm aperf
The Actual Performance Frequency Clock Count
MSR of the core the counter is bound to.
+.It Cm perf-irperf , Cm irperf
+The Instructions Retired Performance Count
+MSR of the core the counter is bound to.
+This event is only available on AMD processors; it is not
+supported on Intel processors.
+See
+.Sx Availability
+below for the full set of conditions that must hold for this
+event to be present.
.El
.Ss Counter Scope
-.Va MPERF
+.Va MPERF ,
+.Va APERF ,
and
-.Va APERF
-are per-core registers: each core has its own independent pair of
+.Va IRPERF
+are per-core registers: each core has its own independent set of
MSRs, and a PMC of this class always reads the value local to the
CPU it is bound to.
Readings from different cores must not be combined or averaged
@@ -108,10 +130,47 @@ class is not registered and does not appear in the output of
.Xr pmccontrol 8
or
.Xr pmcstat 8 .
+.Pp
+The
+.Cm perf-irperf
+event is further restricted to AMD processors that advertise
+support for the IRPERF MSR in CPUID 8000_0008H bit 1 EBX.
+On processors that do not advertise this bit, including all
+Intel processors, the
+.Cm perf-irperf
+event specifier is not available even though the rest of the
+.Dv PMC_CLASS_PERF
+class may be present.
+.Pp
+Due to erratum #1054
+.Pq Dq Instructions Retired Performance Counter May Be Inaccurate ,
+IRPERF is additionally left disabled on affected AMD Zen 1 parts
+.Pq Family 17h, Models below 30h ,
+even though those processors advertise CPUID support for the MSR.
+It is enabled only for Zen 1 Family 17h Model 30h and later, and for
+all subsequent microarchitectures.
+.Pp
+As with
+.Va MPERF
+and
+.Va APERF ,
+the kernel also verifies, at boot time, that the IRPERF MSR is
+actually incrementing before enabling it; some virtualized
+environments advertise CPUID and erratum-safe family/model
+information without functional MSR support behind it.
+.Pp
+If IRPERF is unavailable for any of the reasons above, the rest of
+the
+.Dv PMC_CLASS_PERF
+class
+.Pq Va MPERF No and Va APERF
+is still registered normally; only the
+.Cm perf-irperf
+event specifier is absent.
.Sh EXAMPLES
-Sample both counters on CPU 0 for five seconds:
+Sample all three counters on CPU 0 for five seconds:
.Bd -literal -offset indent
-pmcstat -c 0 -s perf-mperf -s perf-aperf -w 5
+pmcstat -c 0 -s perf-mperf -s perf-aperf -s perf-irperf -w 5
.Ed
.Sh SEE ALSO
.Xr pmc 3 ,
diff --git a/sys/amd64/amd64/initcpu.c b/sys/amd64/amd64/initcpu.c
index 2d2f5835e3b3..a690e3829e7a 100644
--- a/sys/amd64/amd64/initcpu.c
+++ b/sys/amd64/amd64/initcpu.c
@@ -275,6 +275,7 @@ cpu_init_small_core(void)
void
initializecpu(void)
{
+ uint64_t irperf1, irperf2;
uint64_t msr;
uint64_t cr4;
@@ -360,6 +361,36 @@ initializecpu(void)
(cpu_stdext_feature2 & CPUID_STDEXT2_RDPID) != 0)
wrmsr(MSR_TSC_AUX, cpu_auxmsr());
+ /*
+ * According to erratum #1054 "Instructions Retired Performance
+ * Counter May Be Inaccurate", the IRPERF feature should be
+ * enabled only for a subset of Zen 1 (Family 17h Models 30h and above)
+ * and all later processors.
+ *
+ * Separately, verify IRPERF is actually incrementing across two
+ * back-to-back reads. This only catches a dead/non-functional MSR
+ * (e.g. a hypervisor or emulator exposing the CPUID feature bit
+ * without real MSR support behind it) -- it cannot detect the
+ * erratum's actual failure mode, a counter that increments but
+ * returns inaccurate values. That case is handled entirely by the
+ * family/model check above.
+ *
+ * This runs on every core (BSP and each AP) via initializecpu();
+ * pmc_amd_initialize() later assumes IRPERF's enabled/disabled state
+ * is consistent across all cores.
+ */
+ if ((amd_extended_feature_extensions & AMDFEID_IRPERF) &&
+ ((CPUID_TO_FAMILY(cpu_id) > 0x17) ||
+ (CPUID_TO_FAMILY(cpu_id) == 0x17 && CPUID_TO_MODEL(cpu_id) >= 0x30))) {
+ wrmsr(MSR_HWCR, rdmsr(MSR_HWCR) | HWCR_IRPERFEN);
+
+ irperf1 = rdmsr(MSR_IRPERF);
+ irperf2 = rdmsr(MSR_IRPERF);
+
+ if (irperf2 <= irperf1)
+ wrmsr(MSR_HWCR, rdmsr(MSR_HWCR) & ~HWCR_IRPERFEN);
+ }
+
if (!IS_BSP())
cpu_init_small_core();
TSEXIT();
diff --git a/sys/amd64/include/pmc_mdep.h b/sys/amd64/include/pmc_mdep.h
index 8eccff3dfa30..f073d1f7594c 100644
--- a/sys/amd64/include/pmc_mdep.h
+++ b/sys/amd64/include/pmc_mdep.h
@@ -67,7 +67,7 @@ struct pmc_mdep;
* TSC The timestamp counter
* K8 AMD Athlon64 and Opteron PMCs in 64 bit mode.
* IBS AMD IBS
- * PERF AMD/Intel MPERF and APERF
+ * PERF AMD/Intel MPERF, APERF and IRPERF
* IAP Intel Core/Core2/Atom CPUs in 64 bits mode.
* IAF Intel fixed-function PMCs in Core2 and later CPUs.
* UCP Intel Uncore programmable PMCs.
diff --git a/sys/dev/hwpmc/hwpmc_amd.c b/sys/dev/hwpmc/hwpmc_amd.c
index 5de283946384..2231824df6a8 100644
--- a/sys/dev/hwpmc/hwpmc_amd.c
+++ b/sys/dev/hwpmc/hwpmc_amd.c
@@ -1643,8 +1643,21 @@ pmc_amd_initialize(void)
* cpu_startup() -> startrtclock()), not by hwpmc's TSC PMC class. It is
* set only after confirming both MSRs actually increment (some emulators
* expose the CPUID bit without real MSR support).
+ *
+ * Separately, detect support for the IRPERF MSR (AMD-only). If the
+ * feature is present, HWCR_IRPERFEN should already be set in MSR_HWCR
+ * by initializecpu() at boot, which also accounts for erratum #1054
+ * (IRPERF is left disabled on affected Zen 1 parts) and verifies IRPERF
+ * is actually incrementing on a dead/emulated MSR. This code only reads
+ * back that already-decided state.
+ *
+ * Either condition above causes the PERF class to be registered, so at
+ * least one of MPERF/APERF or IRPERF is expected to be usable when this
+ * branch is taken.
*/
- if ((cpu_power_ecx & CPUID_PERF_STAT) && (tsc_perf_stat == 1)) {
+ if (((cpu_power_ecx & CPUID_PERF_STAT) && (tsc_perf_stat == 1)) ||
+ ((amd_extended_feature_extensions & AMDFEID_IRPERF) &&
+ (rdmsr(MSR_HWCR) & HWCR_IRPERFEN))) {
nclasses++;
}
diff --git a/sys/dev/hwpmc/hwpmc_perf.c b/sys/dev/hwpmc/hwpmc_perf.c
index c6f271947829..db7d89b70549 100644
--- a/sys/dev/hwpmc/hwpmc_perf.c
+++ b/sys/dev/hwpmc/hwpmc_perf.c
@@ -3,10 +3,11 @@
*
* SPDX-License-Identifier: BSD-2-Clause
*
- * AMD/Intel MPERF and APERF MSRs counters exposed as an hwpmc(4) PMC class.
+ * AMD/Intel MPERF/APERF and AMD-only IRPERF MSR counters exposed as an
+ * hwpmc(4) PMC class.
*
* Read-only, system-scope (PMC_MODE_SC), 64-bit counters reporting
- * MPERF and APERF MSR values.
+ * MPERF/APERF and IRPERF MSR values.
*/
#include <sys/param.h>
@@ -18,31 +19,18 @@
#define PERF_CAPS PMC_CAP_READ
+static int perf_npmcs = 0;
+
struct perf_descr {
struct pmc_descr pm_descr; /* "base class" */
+ enum pmc_event pm_ev;
+ u_int pm_msr;
};
-static const struct perf_descr perf_pmcdesc[PERF_NPMCS] = {
- {
- .pm_descr = {
- .pd_name = "MPERF",
- .pd_class = PMC_CLASS_PERF,
- .pd_caps = PERF_CAPS,
- .pd_width = 64
- },
- },
- {
- .pm_descr = {
- .pd_name = "APERF",
- .pd_class = PMC_CLASS_PERF,
- .pd_caps = PERF_CAPS,
- .pd_width = 64
- }
- }
-};
+static struct perf_descr perf_pmcdesc[PERF_NPMCS_TOTAL];
struct perf_cpu {
- struct pmc_hw tc_hw[PERF_NPMCS];
+ struct pmc_hw tc_hw[PERF_NPMCS_TOTAL];
};
static struct perf_cpu **perf_pcpu;
@@ -54,7 +42,7 @@ perf_allocate_pmc(int cpu __diagused, int ri __diagused,
{
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU value %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS,
+ KASSERT(ri >= 0 && ri < perf_npmcs,
("[perf,%d] illegal row index %d", __LINE__, ri));
if (a->pm_class != PMC_CLASS_PERF)
@@ -69,7 +57,6 @@ perf_allocate_pmc(int cpu __diagused, int ri __diagused,
* Frequency is an indirect power proxy and could be abused by
* attacks like Hertzbleed.
*/
-
if (priv_check(curthread, PRIV_PMC_SYSTEM) != 0)
return (EPERM);
@@ -78,18 +65,8 @@ perf_allocate_pmc(int cpu __diagused, int ri __diagused,
if ((a->pm_caps & ~PERF_CAPS) != 0)
return (EPERM);
- switch (ri) {
- case PERF_MPERF:
- if (a->pm_ev != PMC_EV_PERF_MPERF)
- return (EINVAL);
- break;
- case PERF_APERF:
- if (a->pm_ev != PMC_EV_PERF_APERF)
- return (EINVAL);
- break;
- default:
+ if (a->pm_ev != perf_pmcdesc[ri].pm_ev)
return (EINVAL);
- }
return (0);
}
@@ -103,7 +80,7 @@ perf_config_pmc(int cpu, int ri, struct pmc *pm)
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU value %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS, ("[perf,%d] illegal row-index %d",
+ KASSERT(ri >= 0 && ri < perf_npmcs, ("[perf,%d] illegal row-index %d",
__LINE__, ri));
phw = &perf_pcpu[cpu]->tc_hw[ri];
@@ -125,7 +102,7 @@ perf_describe(int cpu, int ri, struct pmc_info *pi, struct pmc **ppmc)
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS, ("[perf,%d] illegal row-index %d",
+ KASSERT(ri >= 0 && ri < perf_npmcs, ("[perf,%d] illegal row-index %d",
__LINE__, ri));
phw = &perf_pcpu[cpu]->tc_hw[ri];
@@ -150,7 +127,7 @@ perf_get_config(int cpu, int ri, struct pmc **ppm)
{
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS, ("[perf,%d] illegal row-index %d",
+ KASSERT(ri >= 0 && ri < perf_npmcs, ("[perf,%d] illegal row-index %d",
__LINE__, ri));
*ppm = perf_pcpu[cpu]->tc_hw[ri].phw_pmc;
@@ -161,7 +138,7 @@ perf_get_config(int cpu, int ri, struct pmc **ppm)
static int
perf_get_msr(int ri __diagused, uint32_t *msr __unused)
{
- KASSERT(ri >= 0 && ri < PERF_NPMCS,
+ KASSERT(ri >= 0 && ri < perf_npmcs,
("[perf,%d] ri %d out of range", __LINE__, ri));
return (EINVAL);
@@ -183,7 +160,7 @@ perf_pcpu_fini(struct pmc_mdep *md, int cpu)
ri = md->pmd_classdep[perf_classindex].pcd_ri;
pc = pmc_pcpu[cpu];
- for (i = 0; i < PERF_NPMCS; i++) {
+ for (i = 0; i < perf_npmcs; i++) {
pc->pc_hwpmcs[i + ri] = NULL;
}
@@ -215,7 +192,7 @@ perf_pcpu_init(struct pmc_mdep *md, int cpu)
KASSERT(pc, ("[perf,%d] null generic per-cpu", __LINE__));
- for (i = 0; i < PERF_NPMCS; i++) {
+ for (i = 0; i < perf_npmcs; i++) {
perf_pc->tc_hw[i].phw_state = PMC_PHW_FLAG_IS_ENABLED |
PMC_PHW_CPU_TO_STATE(cpu) | PMC_PHW_INDEX_TO_STATE(i) |
PMC_PHW_FLAG_IS_SHAREABLE;
@@ -232,7 +209,7 @@ perf_read_pmc(int cpu __diagused, int ri, struct pmc *pm, pmc_value_t *v)
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU value %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS, ("[perf,%d] illegal ri %d",
+ KASSERT(ri >= 0 && ri < perf_npmcs, ("[perf,%d] illegal ri %d",
__LINE__, ri));
mode = PMC_TO_MODE(pm);
@@ -242,16 +219,7 @@ perf_read_pmc(int cpu __diagused, int ri, struct pmc *pm, pmc_value_t *v)
PMCDBG1(MDP, REA, 1, "perf-read id=%d", ri);
- switch (ri) {
- case PERF_MPERF:
- *v = rdmsr(MSR_MPERF);
- break;
- case PERF_APERF:
- *v = rdmsr(MSR_APERF);
- break;
- default:
- return (EINVAL);
- }
+ *v = rdmsr(perf_pmcdesc[ri].pm_msr);
return (0);
}
@@ -262,7 +230,7 @@ perf_write_pmc(int cpu __diagused, int ri __diagused, struct pmc *pm __unused,
{
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU value %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS, ("[perf,%d] illegal row-index %d",
+ KASSERT(ri >= 0 && ri < perf_npmcs, ("[perf,%d] illegal row-index %d",
__LINE__, ri));
return (0);
@@ -273,7 +241,7 @@ perf_release_pmc(int cpu __diagused, int ri __diagused, struct pmc *pmc __unused
{
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU value %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS,
+ KASSERT(ri >= 0 && ri < perf_npmcs,
("[perf,%d] illegal row-index %d", __LINE__, ri));
KASSERT(perf_pcpu[cpu]->tc_hw[ri].phw_pmc == NULL,
("[perf,%d] PHW pmc non-NULL", __LINE__));
@@ -286,7 +254,7 @@ perf_start_pmc(int cpu __diagused, int ri __diagused, struct pmc *pm __unused)
{
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU value %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS, ("[perf,%d] illegal row-index %d",
+ KASSERT(ri >= 0 && ri < perf_npmcs, ("[perf,%d] illegal row-index %d",
__LINE__, ri));
return (0);
@@ -297,7 +265,7 @@ perf_stop_pmc(int cpu __diagused, int ri __diagused, struct pmc *pm __unused)
{
KASSERT(cpu >= 0 && cpu < pmc_cpu_max(),
("[perf,%d] illegal CPU value %d", __LINE__, cpu));
- KASSERT(ri >= 0 && ri < PERF_NPMCS, ("[perf,%d] illegal row-index %d",
+ KASSERT(ri >= 0 && ri < perf_npmcs, ("[perf,%d] illegal row-index %d",
__LINE__, ri));
return (0);
@@ -313,6 +281,46 @@ pmc_perf_initialize(struct pmc_mdep *md, int maxcpu, int classindex)
__LINE__, md->pmd_nclass));
if ((cpu_power_ecx & CPUID_PERF_STAT) && (tsc_perf_stat == 1)) {
+ strlcpy(perf_pmcdesc[perf_npmcs].pm_descr.pd_name, "MPERF",
+ PMC_NAME_MAX);
+
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_class = PMC_CLASS_PERF;
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_caps = PERF_CAPS;
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_width = 64;
+ perf_pmcdesc[perf_npmcs].pm_msr = MSR_MPERF;
+ perf_pmcdesc[perf_npmcs].pm_ev = PMC_EV_PERF_MPERF;
+ perf_npmcs++;
+
+ strlcpy(perf_pmcdesc[perf_npmcs].pm_descr.pd_name, "APERF",
+ PMC_NAME_MAX);
+
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_class = PMC_CLASS_PERF;
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_caps = PERF_CAPS;
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_width = 64;
+ perf_pmcdesc[perf_npmcs].pm_msr = MSR_APERF;
+ perf_pmcdesc[perf_npmcs].pm_ev = PMC_EV_PERF_APERF;
+ perf_npmcs++;
+ }
+
+ /*
+ * The IRPERF MSR is supported only for AMD CPUs. The feature is
+ * enabled during the boot on each CPU available if AMDFEID_IRPERF
+ * is set on amd_extended_feature_extensions.
+ */
+ if ((amd_extended_feature_extensions & AMDFEID_IRPERF) &&
+ (rdmsr(MSR_HWCR) & HWCR_IRPERFEN)) {
+ strlcpy(perf_pmcdesc[perf_npmcs].pm_descr.pd_name, "IRPERF",
+ PMC_NAME_MAX);
+
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_class = PMC_CLASS_PERF;
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_caps = PERF_CAPS;
+ perf_pmcdesc[perf_npmcs].pm_descr.pd_width = 64;
+ perf_pmcdesc[perf_npmcs].pm_msr = MSR_IRPERF;
+ perf_pmcdesc[perf_npmcs].pm_ev = PMC_EV_PERF_IRPERF;
+ perf_npmcs++;
+ }
+
+ if (perf_npmcs) {
perf_pcpu = malloc(sizeof(struct perf_cpu *) * maxcpu, M_PMC,
M_ZERO | M_WAITOK);
@@ -320,9 +328,9 @@ pmc_perf_initialize(struct pmc_mdep *md, int maxcpu, int classindex)
perf_classindex = classindex;
pcd = &md->pmd_classdep[classindex];
- pcd->pcd_caps = PMC_CAP_READ;
+ pcd->pcd_caps = PERF_CAPS;
pcd->pcd_class = PMC_CLASS_PERF;
- pcd->pcd_num = PERF_NPMCS;
+ pcd->pcd_num = perf_npmcs;
pcd->pcd_ri = md->pmd_npmc;
pcd->pcd_width = 64;
@@ -339,7 +347,7 @@ pmc_perf_initialize(struct pmc_mdep *md, int maxcpu, int classindex)
pcd->pcd_start_pmc = perf_start_pmc;
pcd->pcd_stop_pmc = perf_stop_pmc;
- md->pmd_npmc += PERF_NPMCS;
+ md->pmd_npmc += perf_npmcs;
}
return (0);
diff --git a/sys/dev/hwpmc/hwpmc_perf.h b/sys/dev/hwpmc/hwpmc_perf.h
index 855c91d290f9..0fc2dfbe70fc 100644
--- a/sys/dev/hwpmc/hwpmc_perf.h
+++ b/sys/dev/hwpmc/hwpmc_perf.h
@@ -3,21 +3,21 @@
*
* SPDX-License-Identifier: BSD-2-Clause
*
- * AMD/Intel MPERF and APERF MSRs counters exposed as an hwpmc(4) PMC class.
+ * AMD/Intel MPERF/APERF MSR counters, and the AMD-only IRPERF counter,
+ * exposed as an hwpmc(4) PMC class.
*/
#ifdef _KERNEL
/*
- * A row per MSR: MPERF and APERF.
+ * A row per MSR: MPERF, APERF and IRPERF.
*/
-#define PERF_NPMCS 2
-#define PERF_MPERF 0
-#define PERF_APERF 1
+#define PERF_NPMCS_TOTAL 3
-extern int tsc_perf_stat;
-extern u_int cpu_power_ecx;
+extern int tsc_perf_stat;
+extern u_int cpu_power_ecx;
+extern u_int amd_extended_feature_extensions;
/*
* Prototypes.
diff --git a/sys/dev/hwpmc/pmc_events.h b/sys/dev/hwpmc/pmc_events.h
index 442c1e0a0fe0..e49bf6d99fc6 100644
--- a/sys/dev/hwpmc/pmc_events.h
+++ b/sys/dev/hwpmc/pmc_events.h
@@ -62,13 +62,14 @@ __PMC_EV_ALIAS("cycles", TSC_TSC)
#define PMC_EV_RAPL_FIRST PMC_EV_RAPL_ENERGY_PKG
#define PMC_EV_RAPL_LAST PMC_EV_RAPL_ENERGY_DRAM
-/* MPERF / APERF MSRs */
+/* MPERF / APERF / IRPERF MSRs */
#define __PMC_EV_PERF() \
__PMC_EV(PERF, MPERF) \
- __PMC_EV(PERF, APERF)
+ __PMC_EV(PERF, APERF) \
+ __PMC_EV(PERF, IRPERF)
#define PMC_EV_PERF_FIRST PMC_EV_PERF_MPERF
-#define PMC_EV_PERF_LAST PMC_EV_PERF_APERF
+#define PMC_EV_PERF_LAST PMC_EV_PERF_IRPERF
/*
* Software events are dynamically defined.
diff --git a/sys/x86/include/specialreg.h b/sys/x86/include/specialreg.h
index 32d672522965..d51d826fc6f7 100644
--- a/sys/x86/include/specialreg.h
+++ b/sys/x86/include/specialreg.h
@@ -110,6 +110,28 @@
#define EFER_UAIE 0x000100000 /* Upper Address Ignore */
#define EFER_AIBRSE 0x000200000 /* Automatic IBRS */
+/*
+ * Bits in AMD64 special registers. HWCR is 64 bits wide
+ */
+#define HWCR_SMMLOCK 0x00000001 /* SMM code lock (R/W) */
+#define HWCR_TLBCACHEDIS 0x00000008 /* Cacheable memory disable (R/W) */
+#define HWCR_INVDWBINVD 0x00000010 /* INVD to WBINVD conversion (R/W) */
+#define HWCR_ALLOWFERRONNE 0x00000080 /* Allow FERR on NE (R/W) */
+#define HWCR_IGNNEEM 0x00000100 /* IGNNE port emulation enable (R/W) */
+#define HWCR_MONMWAITDIS 0x00000200 /* MONITOR and MWAIT disable (R/W) */
+#define HWCR_MONMWAITUSEREN 0x00000400 /* MONITOR/MWAIT user mode enable (R/W) */
+#define HWCR_SMISPCYCDIS 0x00002000 /* SMI special bus cycle disable */
+#define HWCR_RSMSPCYCDIS 0x00004000 /* RSM special bus cycle disable */
+#define HWCR_WRAP32DIS 0x00020000 /* 32-bit address wrap disable (R/W) */
+#define HWCR_MCSTATUSWREN 0x00040000 /* Machine check status write enable (R/W) */
+#define HWCR_IOCFGGPFAULT 0x00100000 /* IO-space configuration causes a GP fault (R/W) */
+#define HWCR_LOCKTSCTOCURRENTP0 0x00200000 /* Lock the TSC to the current P0 frequency (R/W) */
+#define HWCR_TSCFREQSEL 0x01000000 /* TSC frequency select (R) */
+#define HWCR_CPBDIS 0x02000000 /* Core performance boost disable (R/W) */
+#define HWCR_EFFFREQCNTMWAIT 0x04000000 /* Effective frequency counting during mwait (R/W) */
+#define HWCR_EFFFREQREADONLYLOCK 0x08000000 /* Read-only effective frequency counter lock */
+#define HWCR_IRPERFEN 0x40000000 /* Enable instructions retired counter (R/W) */
+
/*
* Intel Extended Features registers
*/
@@ -1253,6 +1275,7 @@
#define MSR_LSTAR 0xc0000082 /* long mode SYSCALL target rip */
#define MSR_CSTAR 0xc0000083 /* compat mode SYSCALL target rip */
#define MSR_SF_MASK 0xc0000084 /* syscall flags mask */
+#define MSR_IRPERF 0xc00000e9 /* instructions retired performance count */
#define MSR_FSBASE 0xc0000100 /* base address of the %fs "segment" */
#define MSR_GSBASE 0xc0000101 /* base address of the %gs "segment" */
#define MSR_KGSBASE 0xc0000102 /* base address of the kernel %gs */