From nobody Tue Jan 03 10:17:34 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NmTFk4r4Qz2pNKw for ; Tue, 3 Jan 2023 10:18:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4NmTFk1h5Vz4JJL for ; Tue, 3 Jan 2023 10:18:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 303AHYVu055757 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 3 Jan 2023 12:17:37 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 303AHYVu055757 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 303AHY8r055756; Tue, 3 Jan 2023 12:17:34 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 3 Jan 2023 12:17:34 +0200 From: Konstantin Belousov To: "Chen, Alvin W" Cc: Amar Takhar , "freebsd-current@freebsd.org" Subject: Re: Status of Intel Hybrid CPU support (Alder Lake/Raptor Lake) support Message-ID: References: <20221214201823.GA843@darkbeer.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on tom.home X-Rspamd-Queue-Id: 4NmTFk1h5Vz4JJL X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Tue, Jan 03, 2023 at 08:20:16AM +0000, Chen, Alvin W wrote: > > > > [EXTERNAL EMAIL] > > > > On 2022-11-14 09:09 +0200, Konstantin Belousov wrote: > > > > > > You might use this patch meantime > > > https://urldefense.com/v3/__https://kib.kiev.ua/git/gitweb.cgi?p=devia > > > > > nt3.git;a=commit;h=5d72240a8777b26d5e0a7d2d26bb919d05f60002__;!!Lp > > KI!j > > > > > pyHChyB8NZAQq5isiNFepD61cX0HczrFCeOriYbSwyfMPGW7k8I_BYxPWXv1FG > > yfl1Y-Ip > > > dgcNkyg$ [kib[.]kiev[.]ua] > > > > Thank you for this patch. On my 13.1 machine it fixed all the panics with a i9- > > 12900KF. It's nice to be able to use all the cores since I got this machine in > > May. I had them disabled in BIOS. > > > > Also as a side note this fixed the audio issues I was having with the e cores > > both enabled and disabled. > > > > This patch can't work well with some older CPUs, and I find one N6005 (Intel JSP). > This JSP cpu is not a hybrid cpu architecture, but the high cpu id is 0x1b with 0x20000000 which meet the small core condition in the patch, but kernel got crashed on this cpu. > We need a better condition to pick up the E cores to make sure the kernel can be compatible with all older Non-Hybrid CPUs. How does the kernel panic? Could you show the verbose dmesg lines with CPUID information? If this is JasperLake, I suspect that the patch below should be enough. Still, the workaround is not confirmed/explained by you (Intel). I have a high hope that we would eventually be able to use INVLPG with PCID on (newer) small cores, might be even on older small cores after a microcode update. commit c8812792202d73804cccd624f6478c4069b22438 Author: Konstantin Belousov Date: Tue Jan 3 12:13:07 2023 +0200 amd64: be more precise when enabling the AlderLake workaround Reported by: "Chen, Alvin W" Sponsored by: The FreeBSD Foundation MFC after: 1 week diff --git a/sys/amd64/amd64/initcpu.c b/sys/amd64/amd64/initcpu.c index 08385d3095d0..cddf8502437e 100644 --- a/sys/amd64/amd64/initcpu.c +++ b/sys/amd64/amd64/initcpu.c @@ -247,6 +247,26 @@ cpu_auxmsr(void) return (PCPU_GET(cpuid)); } +void +cpu_init_small_core(void) +{ + u_int r[4]; + + if (cpu_high < 0x1a) + return; + + cpuid_count(0x1a, 0, r); + if ((r[0] & CPUID_HYBRID_CORE_MASK) != CPUID_HYBRID_SMALL_CORE) + return; + + PCPU_SET(small_core, 1); + if (pmap_pcid_enabled && invpcid_works && + pmap_pcid_invlpg_workaround_uena) { + PCPU_SET(pcid_invlpg_workaround, 1); + pmap_pcid_invlpg_workaround = 1; + } +} + /* * Initialize CPU control registers */ @@ -255,7 +275,6 @@ initializecpu(void) { uint64_t msr; uint32_t cr4; - u_int r[4]; cr4 = rcr4(); if ((cpu_feature & CPUID_XMM) && (cpu_feature & CPUID_FXSR)) { @@ -319,18 +338,8 @@ initializecpu(void) (cpu_stdext_feature2 & CPUID_STDEXT2_RDPID) != 0) wrmsr(MSR_TSC_AUX, cpu_auxmsr()); - if (cpu_high >= 0x1a) { - cpuid_count(0x1a, 0, r); - if ((r[0] & CPUID_HYBRID_CORE_MASK) == - CPUID_HYBRID_SMALL_CORE) { - PCPU_SET(small_core, 1); - if (pmap_pcid_enabled && - pmap_pcid_invlpg_workaround_uena) { - PCPU_SET(pcid_invlpg_workaround, 1); - pmap_pcid_invlpg_workaround = 1; - } - } - } + if (!IS_BSP()) + cpu_init_small_core(); } void diff --git a/sys/amd64/amd64/machdep.c b/sys/amd64/amd64/machdep.c index 05342b31d2aa..c601ce868978 100644 --- a/sys/amd64/amd64/machdep.c +++ b/sys/amd64/amd64/machdep.c @@ -1341,6 +1341,12 @@ hammer_time(u_int64_t modulep, u_int64_t physfree) pmap_pcid_enabled = 0; } + /* + * Now we can do small core initialization, after the PCID + * CPU features and user knobs are evaluated. + */ + cpu_init_small_core(); + link_elf_ireloc(kmdp); /* diff --git a/sys/amd64/include/md_var.h b/sys/amd64/include/md_var.h index f014c66c0d06..f5cbdb6bbd9d 100644 --- a/sys/amd64/include/md_var.h +++ b/sys/amd64/include/md_var.h @@ -72,6 +72,7 @@ void amd64_bsp_ist_init(struct pcpu *pc); void amd64_syscall(struct thread *td, int traced); void amd64_syscall_ret_flush_l1d(int error); void amd64_syscall_ret_flush_l1d_recalc(void); +void cpu_init_small_core(void); void doreti_iret(void) __asm(__STRING(doreti_iret)); void doreti_iret_fault(void) __asm(__STRING(doreti_iret_fault)); void flush_l1d_sw_abi(void);