From nobody Tue Jul 19 07:37:17 2022 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Ln9dp0FVhz4WbXT; Tue, 19 Jul 2022 07:37:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ln9dn4jfZz3pbr; Tue, 19 Jul 2022 07:37:17 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1658216237; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bb7ebZ4oxcaft4pZpRVddSJvVHesp/Pib33L8Y8WN7E=; b=O+H7HLpFfmIzgBtnjMVdQurx2l0cw9rRr+Ez5bhmH54zmMP6itsiNC7bEei24GS4An3qIs YPHPz1NjqNnIYnjplOurHkk4Zfaztf+XVVZ7Z46U6ToigXaZrwizbmvV+LbsOO20jFC3YG pZ6qZShwTdxhnZA0VgkLzZyFDWbwdZdeZ+0e7wBfE/A965Tr3+A7NsxtiDGy5vYkKmRS2U 9YdgyxR68kFrJIUjXBWPjdoMJe9RRhj86c0RcdEI/EmhgE/nueTWZwtEto6PhChPBRC1HV 8hRZYNocDC67dUFgd/dXL7e5e/SFAY2u1QgxHwN6HLy2Z0VRlZ/EMnwnu2nzQg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Ln9dn3Lb3z19XB; Tue, 19 Jul 2022 07:37:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 26J7bH0c021406; Tue, 19 Jul 2022 07:37:17 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 26J7bHBp021405; Tue, 19 Jul 2022 07:37:17 GMT (envelope-from git) Date: Tue, 19 Jul 2022 07:37:17 GMT Message-Id: <202207190737.26J7bHBp021405@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: c553f62a3007 - stable/13 - Add arm64 CnP support List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: c553f62a300735ceaf1c3ca6a9a012e78a69e662 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1658216237; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bb7ebZ4oxcaft4pZpRVddSJvVHesp/Pib33L8Y8WN7E=; b=JXDM4dbM8mRBoT2DdAf2M07Y+Zy7pE9QKxzbBPnACLLUWP4lvQWWnc67OhrEBV3fmobyw0 ZbJWy8SgEd/BBzD1gSb+k7bu8bq/xkg9HqSlfMmaoXs07hM7Ldg6VfLvewT6+3NLhgZh/o t4Ptt9qeI1e+Y1nLtmkmXYXLu+Fyl32ouk3JCxe3AnQU4uJHm2HJXNlQ3Hr3Wwhg0NkeYB MZsg6C0fAvApm2ihvQ/c63aJX2ZmPbtBgcNvycklf5f5puYWU2Rvyb1gl3Mg4V/iCNPXmK 4c4Vu7DaWVsJxy07INh3UsIR7EcobrvBu+yOpTeUW3RkfNmp5nbbZeyxkMeKuQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1658216237; a=rsa-sha256; cv=none; b=PUHA5ZglD9B82FSSEoh5aHbK/rtVL/Vqcgzyq4Imfy2oNxoN/KZ0b+GBmhr568K4TlacBJ R9sBNAv2Vii211kURR2o6oVyPWBC51nGG2V1WHXG6oOJtHQUMRdNX/jTW7PiKAEjgWpmFq RsLuKySzA6FXBskIIQAiUl6W4QTUqYTsVX6ZyBOpFjEDTSp23I0hHGfsC5tIaUn9hNJkc/ MgWGlpI2gMlbaEevxPpsvcqryHfedgXww4oH4HrSz3lKGwIQUdbIEtSQh/nDOqdSlj3rnA Tfz13DsUqkRIhOMnSZyQzqveMUb2Qho18PgF913trh5h2XR1zLGghRyx6VNidw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=c553f62a300735ceaf1c3ca6a9a012e78a69e662 commit c553f62a300735ceaf1c3ca6a9a012e78a69e662 Author: Andrew Turner AuthorDate: 2022-05-25 09:55:40 +0000 Commit: Andrew Turner CommitDate: 2022-07-19 07:03:30 +0000 Add arm64 CnP support Set the Common not Private bit in the ttbr registers when supported on arm64. This tells the hardware it can share the translation table entries on multiple CPUs. Reviewed by: alc, kib Sponsored by: The FreeBSD Foundation (cherry picked from commit 9e0716f4610158936c0b9cec1a9930c4ebe924fb) --- sys/arm64/arm64/pmap.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 2 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 0c6f3630c42d..a8615b0a3902 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -6845,6 +6845,8 @@ out: mtx_unlock_spin(&set->asid_set_mutex); } +static uint64_t __read_mostly ttbr_flags; + /* * Compute the value that should be stored in ttbr0 to activate the specified * pmap. This value may change from time to time. @@ -6852,10 +6854,70 @@ out: uint64_t pmap_to_ttbr0(pmap_t pmap) { + uint64_t ttbr; + + ttbr = pmap->pm_ttbr; + ttbr |= ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); + ttbr |= ttbr_flags; + + return (ttbr); +} + +static void +pmap_set_cnp(void *arg) +{ + uint64_t ttbr0, ttbr1; + u_int cpuid; + + cpuid = *(u_int *)arg; + if (cpuid == curcpu) { + /* + * Set the flags while all CPUs are handling the + * smp_rendezvous so will not call pmap_to_ttbr0. Any calls + * to pmap_to_ttbr0 after this will have the CnP flag set. + * The dsb after invalidating the TLB will act as a barrier + * to ensure all CPUs can observe this change. + */ + ttbr_flags |= TTBR_CnP; + } + + ttbr0 = READ_SPECIALREG(ttbr0_el1); + ttbr0 |= TTBR_CnP; + + ttbr1 = READ_SPECIALREG(ttbr1_el1); + ttbr1 |= TTBR_CnP; + + /* Update ttbr{0,1}_el1 with the CnP flag */ + WRITE_SPECIALREG(ttbr0_el1, ttbr0); + WRITE_SPECIALREG(ttbr1_el1, ttbr1); + isb(); + __asm __volatile("tlbi vmalle1is"); + dsb(ish); + isb(); +} + +/* + * Defer enabling CnP until we have read the ID registers to know if it's + * supported on all CPUs. + */ +static void +pmap_init_cnp(void *dummy __unused) +{ + uint64_t reg; + u_int cpuid; + + if (!get_kernel_reg(ID_AA64MMFR2_EL1, ®)) + return; + + if (ID_AA64MMFR2_CnP_VAL(reg) != ID_AA64MMFR2_CnP_NONE) { + if (bootverbose) + printf("Enabling CnP\n"); + cpuid = curcpu; + smp_rendezvous(NULL, pmap_set_cnp, NULL, &cpuid); + } - return (ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)) | - pmap->pm_ttbr); } +SYSINIT(pmap_init_cnp, SI_SUB_SMP, SI_ORDER_ANY, pmap_init_cnp, NULL); static bool pmap_activate_int(pmap_t pmap)