From nobody Thu Mar 31 16:07:49 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DF4D91A3DE40; Thu, 31 Mar 2022 16:07:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KTp9d5vRyz4Yh0; Thu, 31 Mar 2022 16:07:49 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648742869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vOi+LreYrezvdbJfNICCRxRcfowZG9mEoImWr2i+mlI=; b=AbQTH5LUXk4DrE827dlhy5kvN1qwMQNVeAdOrQxnSmdQJWtfTtiNeMl1MULCx3I0WvQEoO oY3werzQ2p78Am4o48GH90mhaMaChvgfYHulGWiahO/xlW80BLvLPSOJBHvqMvO1s0+Gv6 ZBL7xCQHWGoEqrQpTfgZllGCYyq3p3vanzphrhPO/BXF1Wq69NySxIZjrre2fiLh9ffqpY HEDbuoQmK8Ep+NINM2+ou+CqYEmHRhEdarJl6Aa6XEHfN7iGhR8NfeUUPZ9YGf0ld1kvJI taDnpcQ+Wh4hpBPTOSIM7yZ3czyfZC6Guito87FQ+a+Q4zQs/i2KsavOd3hU4w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A70B42AB26; Thu, 31 Mar 2022 16:07:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 22VG7nLc095127; Thu, 31 Mar 2022 16:07:49 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 22VG7nZd095126; Thu, 31 Mar 2022 16:07:49 GMT (envelope-from git) Date: Thu, 31 Mar 2022 16:07:49 GMT Message-Id: <202203311607.22VG7nZd095126@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 44eddec48e5f - stable/13 - x86: Probe the TSC frequency earlier List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 44eddec48e5fc09679ce12210e3a02dfeb14889c Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648742869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vOi+LreYrezvdbJfNICCRxRcfowZG9mEoImWr2i+mlI=; b=eCWK0E6kE/lSOEmoNA7jXkC/e8co6pNI+RpDPOE6ZqBL9vtMod/oxHXD+zeD9q+nxELf4v ofLpxiTTp06b5HKjdthbtHZ+VNCWau/p8wMYY3PS9UvQDF8GosNLQSHD5N+LQ3VqPosasw 70Uqi0bN/gdtziVk1E+a5ALKb2d63691Dm5QrgqH2D7DgUr3zT5IkRDMO7Bnfux/VkPbyA lE51z/NflzBdJq/ikgjFJnEWh30++5Ktne4D7kzQYjUTZwRKZmFHXmyVI+0HdEhn+KCD/j bX66hjHmRGQB3WyG+99/ye1X6/08zrSXB/6iKoVuhMkroaTgwoc/GEXl2rwERA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1648742869; a=rsa-sha256; cv=none; b=VBWxLJzQfrl57lcHvH383T6ZxMUw3ywnYRiEoMC+smOjUriHsU+CjfFA7TkZQe6xBw58Km A+tdH2jhWdn9+E7zNl93BMxpVRW2ypMwoRawZoj9ViUW31kkNbhrgRrLbthNU+eJsLsjhG vKb5F8+nFLgjEaxOjYf+Ow4MJQBdjMu38piX+l1lIWuUUFWHU6s9yQ2zDBVk+6ER5dZm95 fU78ZgN9k2Uf2ZbB5/jC+HIICS0ziGmHU8nlNDEDqKYTdpBb8t2r78qmt8Ar50iOCzcth+ jePvvAySwbupxiEGdsGTNeFvOd8XbGOg3d8Wfu5XrUwIX62ig34KBwxExOOzrQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=44eddec48e5fc09679ce12210e3a02dfeb14889c commit 44eddec48e5fc09679ce12210e3a02dfeb14889c Author: Mark Johnston AuthorDate: 2022-03-01 14:39:35 +0000 Commit: Mark Johnston CommitDate: 2022-03-31 16:05:25 +0000 x86: Probe the TSC frequency earlier This lets us use the TSC to implement early DELAY, limiting the use of the sometimes-unreliable 8254 PIT. PR: 262155 Reviewed by: emaste Tested by: emaste, mike tancsa , Stefan Hegnauer Sponsored by: The FreeBSD Foundation (cherry picked from commit 84369dd52369cbae28970dca20a53d3de1719907) --- sys/amd64/amd64/machdep.c | 14 +++++- sys/i386/i386/machdep.c | 11 ++++- sys/x86/include/clock.h | 3 +- sys/x86/isa/clock.c | 4 +- sys/x86/x86/tsc.c | 123 +++++++++++++++++++++++++--------------------- 5 files changed, 94 insertions(+), 61 deletions(-) diff --git a/sys/amd64/amd64/machdep.c b/sys/amd64/amd64/machdep.c index 0e2e0e9db9e8..870b113359ae 100644 --- a/sys/amd64/amd64/machdep.c +++ b/sys/amd64/amd64/machdep.c @@ -168,6 +168,9 @@ extern u_int64_t hammer_time(u_int64_t, u_int64_t); static void cpu_startup(void *); SYSINIT(cpu, SI_SUB_CPU, SI_ORDER_FIRST, cpu_startup, NULL); +/* Probe 8254 PIT and TSC. */ +static void native_clock_source_init(void); + /* Preload data parse function */ static caddr_t native_parse_preload_data(u_int64_t); @@ -176,8 +179,8 @@ static void native_parse_memmap(caddr_t, vm_paddr_t *, int *); /* Default init_ops implementation. */ struct init_ops init_ops = { - .parse_preload_data = native_parse_preload_data, - .early_clock_source_init = i8254_init, + .parse_preload_data = native_parse_preload_data, + .early_clock_source_init = native_clock_source_init, .early_delay = i8254_delay, .parse_memmap = native_parse_memmap, #ifdef SMP @@ -1170,6 +1173,13 @@ native_parse_preload_data(u_int64_t modulep) return (kmdp); } +static void +native_clock_source_init(void) +{ + i8254_init(); + tsc_init(); +} + static void amd64_kdb_init(void) { diff --git a/sys/i386/i386/machdep.c b/sys/i386/i386/machdep.c index 447bbf969e6f..227e7e07549b 100644 --- a/sys/i386/i386/machdep.c +++ b/sys/i386/i386/machdep.c @@ -188,6 +188,8 @@ struct kva_md_info kmi; static struct trapframe proc0_tf; struct pcpu __pcpu[MAXCPU]; +static void i386_clock_source_init(void); + struct mtx icu_lock; struct mem_range_softc mem_range_softc; @@ -198,13 +200,20 @@ extern struct sysentvec elf32_freebsd_sysvec; /* Default init_ops implementation. */ struct init_ops init_ops = { - .early_clock_source_init = i8254_init, + .early_clock_source_init = i386_clock_source_init, .early_delay = i8254_delay, #ifdef DEV_APIC .msi_init = msi_init, #endif }; +static void +i386_clock_source_init(void) +{ + i8254_init(); + tsc_init(); +} + static void cpu_startup(dummy) void *dummy; diff --git a/sys/x86/include/clock.h b/sys/x86/include/clock.h index 83c8351ed31c..9aeccadf89aa 100644 --- a/sys/x86/include/clock.h +++ b/sys/x86/include/clock.h @@ -28,6 +28,7 @@ void i8254_init(void); void i8254_delay(int); void clock_init(void); void lapic_calibrate(void); +void tsc_init(void); void tsc_calibrate(void); /* @@ -35,7 +36,7 @@ void tsc_calibrate(void); */ void startrtclock(void); -void init_TSC(void); +void start_TSC(void); void resume_TSC(void); #define HAS_TIMER_SPKR 1 diff --git a/sys/x86/isa/clock.c b/sys/x86/isa/clock.c index 1178d35979c1..f21f847709cd 100644 --- a/sys/x86/isa/clock.c +++ b/sys/x86/isa/clock.c @@ -398,10 +398,10 @@ i8254_init(void) } void -startrtclock() +startrtclock(void) { - init_TSC(); + start_TSC(); } void diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c index e0c23767573e..024800077e86 100644 --- a/sys/x86/x86/tsc.c +++ b/sys/x86/x86/tsc.c @@ -265,17 +265,42 @@ tsc_freq_8254(uint64_t *res) static void probe_tsc_freq(void) { - if (cpu_power_ecx & CPUID_PERF_STAT) { - /* - * XXX Some emulators expose host CPUID without actual support - * for these MSRs. We must test whether they really work. - */ - wrmsr(MSR_MPERF, 0); - wrmsr(MSR_APERF, 0); - DELAY(10); - if (rdmsr(MSR_MPERF) > 0 && rdmsr(MSR_APERF) > 0) - tsc_perf_stat = 1; +#ifdef __i386__ + /* The TSC is known to be broken on certain CPUs. */ + switch (cpu_vendor_id) { + case CPU_VENDOR_AMD: + switch (cpu_id & 0xFF0) { + case 0x500: + /* K5 Model 0 */ + tsc_disabled = 1; + return; + } + break; + case CPU_VENDOR_CENTAUR: + switch (cpu_id & 0xff0) { + case 0x540: + /* + * http://www.centtech.com/c6_data_sheet.pdf + * + * I-12 RDTSC may return incoherent values in EDX:EAX + * I-13 RDTSC hangs when certain event counters are used + */ + tsc_disabled = 1; + return; + } + break; + case CPU_VENDOR_NSC: + switch (cpu_id & 0xff0) { + case 0x540: + if ((cpu_id & CPUID_STEPPING) == 0) { + tsc_disabled = 1; + return; + } + break; + } + break; } +#endif switch (cpu_vendor_id) { case CPU_VENDOR_AMD: @@ -315,15 +340,18 @@ probe_tsc_freq(void) break; } - if (tsc_freq_cpuid_vm()) - return; - - if (vm_guest == VM_GUEST_VMWARE) { + if (tsc_freq_cpuid_vm()) { + if (bootverbose) + printf( + "Early TSC frequency %juHz derived from hypervisor CPUID\n", + (uintmax_t)tsc_freq); + } else if (vm_guest == VM_GUEST_VMWARE) { tsc_freq_vmware(); - return; - } - - if (tsc_freq_cpuid(&tsc_freq)) { + if (bootverbose) + printf( + "Early TSC frequency %juHz derived from VMWare hypercall\n", + (uintmax_t)tsc_freq); + } else if (tsc_freq_cpuid(&tsc_freq)) { /* * If possible, use the value obtained from CPUID as the initial * frequency. This will be refined later during boot but is @@ -361,50 +389,26 @@ probe_tsc_freq(void) "Early TSC frequency %juHz calibrated from 8254 PIT\n", (uintmax_t)tsc_freq); } + + if (cpu_power_ecx & CPUID_PERF_STAT) { + /* + * XXX Some emulators expose host CPUID without actual support + * for these MSRs. We must test whether they really work. + */ + wrmsr(MSR_MPERF, 0); + wrmsr(MSR_APERF, 0); + DELAY(10); + if (rdmsr(MSR_MPERF) > 0 && rdmsr(MSR_APERF) > 0) + tsc_perf_stat = 1; + } } void -init_TSC(void) +start_TSC(void) { - if ((cpu_feature & CPUID_TSC) == 0 || tsc_disabled) return; -#ifdef __i386__ - /* The TSC is known to be broken on certain CPUs. */ - switch (cpu_vendor_id) { - case CPU_VENDOR_AMD: - switch (cpu_id & 0xFF0) { - case 0x500: - /* K5 Model 0 */ - return; - } - break; - case CPU_VENDOR_CENTAUR: - switch (cpu_id & 0xff0) { - case 0x540: - /* - * http://www.centtech.com/c6_data_sheet.pdf - * - * I-12 RDTSC may return incoherent values in EDX:EAX - * I-13 RDTSC hangs when certain event counters are used - */ - return; - } - break; - case CPU_VENDOR_NSC: - switch (cpu_id & 0xff0) { - case 0x540: - if ((cpu_id & CPUID_STEPPING) == 0) - return; - break; - } - break; - } -#endif - - probe_tsc_freq(); - /* * Inform CPU accounting about our boot-time clock rate. This will * be updated if someone loads a cpufreq driver after boot that @@ -706,6 +710,15 @@ tsc_update_freq(uint64_t new_freq) new_freq >> (int)(intptr_t)tsc_timecounter.tc_priv); } +void +tsc_init(void) +{ + if ((cpu_feature & CPUID_TSC) == 0 || tsc_disabled) + return; + + probe_tsc_freq(); +} + /* * Perform late calibration of the TSC frequency once ACPI-based timecounters * are available. At this point timehands are not set up, so we read the