From nobody Wed Jun 21 02:39:34 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qm74k4QgDz4gJZW; Wed, 21 Jun 2023 02:39:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qm74k3S5Jz3CRJ; Wed, 21 Jun 2023 02:39:34 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687315174; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Axfp+g/Is4Oz4BcBd0dTN3yQhi1dQcNK7D/i6EazLIg=; b=lTmDdTF0rRlnz9ZZEeuF4O268SaUGts/NGQ4rcnpR37mdi9sDkNQ2JNwibeR7bCMsqyOls hzTduLrlA8SqbfZkvsTI1bK8F5cxZvZwLQw7j7p3clieVicNJa2DExIl6kx4IQAr0uZbFf wlJ5+Mp6cXwHPvVdGFPq9PD40wZCpEL2mJgQzjQb8EBWd5CoojHfImXxGGypZg/suoZBzx CYhoCdVRbOI+4a//jYysU2vFZ/P4dipqDFpVt2mpVq2kq5n4cZnw4Ekm7aKds5jcJj2ueC sfWcMKIXZldHcnCjeBtlr4+kaAC7BlEpi0304Kndhgu5jWeGcMKVOHRMCXPTbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687315174; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Axfp+g/Is4Oz4BcBd0dTN3yQhi1dQcNK7D/i6EazLIg=; b=j3W46eYerjr55Htn+Du6Z980rFUxfsqRwFw1RVwjW0rIkf0xeCfvZoeHSNuSmO05Ziz+b+ ECjO+YgfdYjixq4hBS/wD25lr9l8yuZb6ldJBwen9P3G2w/pBg3za38AvnZG5lN9JSeRaC V053WvhggPxkjyY+qXNEm9zgITD3+k1boPXCHc6lM9BbW2y3355TpImpUZ4KGsQGPpokM2 T9WulSyaXurpLv00s59GI7O4fZk/VuiZT6TiPSQ+oCTfIjFP325oxIbBcWgNFry6tjLlTe KQ4U1Pw0iFIB9/aewDtxpTFESxnMSj8jCd3Eo3pTX5N8QaH6JTxtSgjanC67uA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1687315174; a=rsa-sha256; cv=none; b=GyfBLujPsmYB7v5TxsFlWS3c1+fAddpUjsEwWZdgMjvMppDWJ6DtZKy8k/dNgYXDkQgi/n 4UMlVpUGEEu5ZEXK/D41rA8+87gYY4MyQ5Ri1rC1pVzV2lnDo9qFUYIzHqnDgrl+ls7wHH /haVBXBrCRnwILowg2Yq8y+KVReCr9n5vu/l5yJz0SPbB2HcOpDR5Z4yhnEDXDN/gyt/Ei hBDqTcqG4AvcyuwuyMNWgAR1Dgbabt/VZFCpWtUwd90irB1c2gJQg6B+ZMLBzbPFzokwXV QdyGISfrdw/gTZ6n17qr0wYn/+GQprGNpO+97KpGu4gBcEsI82MbktahavZM8Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Qm74k1v4KzPbn; Wed, 21 Jun 2023 02:39:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 35L2dYYH049298; Wed, 21 Jun 2023 02:39:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 35L2dYmi049297; Wed, 21 Jun 2023 02:39:34 GMT (envelope-from git) Date: Wed, 21 Jun 2023 02:39:34 GMT Message-Id: <202306210239.35L2dYmi049297@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kyle Evans Subject: git: 72cf63a259b5 - stable/13 - arm64: gicv3: setup PPIs on all APs after they're online List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 72cf63a259b5038560bd039b9ec3bcb11e0b3d5d Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=72cf63a259b5038560bd039b9ec3bcb11e0b3d5d commit 72cf63a259b5038560bd039b9ec3bcb11e0b3d5d Author: Kyle Evans AuthorDate: 2023-05-15 17:21:45 +0000 Commit: Kyle Evans CommitDate: 2023-06-20 20:11:43 +0000 arm64: gicv3: setup PPIs on all APs after they're online For all PPIs setup earlier than SI_SUB_SMP, PIC_INIT_SECONDARY ends up cleaning these up for each AP as it comes online. Once they're online, we don't currently do anything to make sure they're configured for other APs. Fix it by using smp_rendezvous for the meaty bits of configuring a PPI, which will just do single-thread behavior before APs are online but do the right thing for other CPUs after. While we're here, make sure redistributor config is correct for other APs as they come online in gic_v3_init_secondary. Reported/Tested by: Souradeep Chakrabarti (Microsoft/Hyper-V) Reviewed by: andrew (before slight refactor) (cherry picked from commit 172af24449cd8d34339172d125832b7ecd274213) --- sys/arm64/arm64/gic_v3.c | 157 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 109 insertions(+), 48 deletions(-) diff --git a/sys/arm64/arm64/gic_v3.c b/sys/arm64/arm64/gic_v3.c index 9a65bdd9d585..653fd6e8d748 100644 --- a/sys/arm64/arm64/gic_v3.c +++ b/sys/arm64/arm64/gic_v3.c @@ -712,15 +712,66 @@ gic_v3_map_intr(device_t dev, struct intr_map_data *data, return (error); } +struct gic_v3_setup_periph_args { + device_t dev; + struct intr_irqsrc *isrc; +}; + +static void +gic_v3_setup_intr_periph(void *argp) +{ + struct gic_v3_setup_periph_args *args = argp; + struct intr_irqsrc *isrc = args->isrc; + struct gic_v3_irqsrc *gi = (struct gic_v3_irqsrc *)isrc; + device_t dev = args->dev; + u_int irq = gi->gi_irq; + struct gic_v3_softc *sc = device_get_softc(dev); + uint32_t reg; + + MPASS(irq <= GIC_LAST_SPI); + + /* + * We need the lock for both SGIs and PPIs for an atomic CPU_SET() at a + * minimum, but we also need it below for SPIs. + */ + mtx_lock_spin(&sc->gic_mtx); + + if (isrc->isrc_flags & INTR_ISRCF_PPI) + CPU_SET(PCPU_GET(cpuid), &isrc->isrc_cpu); + + if (irq >= GIC_FIRST_PPI && irq <= GIC_LAST_SPI) { + /* Set the trigger and polarity */ + if (irq <= GIC_LAST_PPI) + reg = gic_r_read(sc, 4, + GICR_SGI_BASE_SIZE + GICD_ICFGR(irq)); + else + reg = gic_d_read(sc, 4, GICD_ICFGR(irq)); + if (gi->gi_trig == INTR_TRIGGER_LEVEL) + reg &= ~(2 << ((irq % 16) * 2)); + else + reg |= 2 << ((irq % 16) * 2); + + if (irq <= GIC_LAST_PPI) { + gic_r_write(sc, 4, + GICR_SGI_BASE_SIZE + GICD_ICFGR(irq), reg); + gic_v3_wait_for_rwp(sc, REDIST); + } else { + gic_d_write(sc, 4, GICD_ICFGR(irq), reg); + gic_v3_wait_for_rwp(sc, DIST); + } + } + + mtx_unlock_spin(&sc->gic_mtx); +} + static int gic_v3_setup_intr(device_t dev, struct intr_irqsrc *isrc, struct resource *res, struct intr_map_data *data) { - struct gic_v3_softc *sc = device_get_softc(dev); struct gic_v3_irqsrc *gi = (struct gic_v3_irqsrc *)isrc; + struct gic_v3_setup_periph_args pargs; enum intr_trigger trig; enum intr_polarity pol; - uint32_t reg; u_int irq; int error; @@ -749,41 +800,18 @@ gic_v3_setup_intr(device_t dev, struct intr_irqsrc *isrc, gi->gi_trig = trig; } - /* - * XXX - In case that per CPU interrupt is going to be enabled in time - * when SMP is already started, we need some IPI call which - * enables it on others CPUs. Further, it's more complicated as - * pic_enable_source() and pic_disable_source() should act on - * per CPU basis only. Thus, it should be solved here somehow. - */ - if (isrc->isrc_flags & INTR_ISRCF_PPI) - CPU_SET(PCPU_GET(cpuid), &isrc->isrc_cpu); - - if (irq >= GIC_FIRST_PPI && irq <= GIC_LAST_SPI) { - mtx_lock_spin(&sc->gic_mtx); - - /* Set the trigger and polarity */ - if (irq <= GIC_LAST_PPI) - reg = gic_r_read(sc, 4, - GICR_SGI_BASE_SIZE + GICD_ICFGR(irq)); - else - reg = gic_d_read(sc, 4, GICD_ICFGR(irq)); - if (trig == INTR_TRIGGER_LEVEL) - reg &= ~(2 << ((irq % 16) * 2)); - else - reg |= 2 << ((irq % 16) * 2); - - if (irq <= GIC_LAST_PPI) { - gic_r_write(sc, 4, - GICR_SGI_BASE_SIZE + GICD_ICFGR(irq), reg); - gic_v3_wait_for_rwp(sc, REDIST); - } else { - gic_d_write(sc, 4, GICD_ICFGR(irq), reg); - gic_v3_wait_for_rwp(sc, DIST); - } - - mtx_unlock_spin(&sc->gic_mtx); + pargs.dev = dev; + pargs.isrc = isrc; + if (isrc->isrc_flags & INTR_ISRCF_PPI) { + /* + * If APs haven't been fired up yet, smp_rendezvous() will just + * execute it on the single CPU and gic_v3_init_secondary() will + * clean up afterwards. + */ + smp_rendezvous(NULL, gic_v3_setup_intr_periph, NULL, &pargs); + } else if (irq >= GIC_FIRST_SPI && irq <= GIC_LAST_SPI) { + gic_v3_setup_intr_periph(&pargs); gic_v3_bind_intr(dev, isrc); } @@ -828,23 +856,50 @@ gic_v3_disable_intr(device_t dev, struct intr_irqsrc *isrc) panic("%s: Unsupported IRQ %u", __func__, irq); } +static void +gic_v3_enable_intr_periph(void *argp) +{ + struct gic_v3_setup_periph_args *args = argp; + struct gic_v3_irqsrc *gi = (struct gic_v3_irqsrc *)args->isrc; + device_t dev = args->dev; + struct gic_v3_softc *sc = device_get_softc(dev); + u_int irq = gi->gi_irq; + + /* SGIs and PPIs in corresponding Re-Distributor */ + gic_r_write(sc, 4, GICR_SGI_BASE_SIZE + GICD_ISENABLER(irq), + GICD_I_MASK(irq)); + gic_v3_wait_for_rwp(sc, REDIST); +} + static void gic_v3_enable_intr(device_t dev, struct intr_irqsrc *isrc) { + struct gic_v3_setup_periph_args pargs; struct gic_v3_softc *sc; struct gic_v3_irqsrc *gi; u_int irq; - sc = device_get_softc(dev); gi = (struct gic_v3_irqsrc *)isrc; irq = gi->gi_irq; + pargs.isrc = isrc; + pargs.dev = dev; if (irq <= GIC_LAST_PPI) { - /* SGIs and PPIs in corresponding Re-Distributor */ - gic_r_write(sc, 4, GICR_SGI_BASE_SIZE + GICD_ISENABLER(irq), - GICD_I_MASK(irq)); - gic_v3_wait_for_rwp(sc, REDIST); - } else if (irq >= GIC_FIRST_SPI && irq <= GIC_LAST_SPI) { + /* + * SGIs only need configured on the current AP. We'll setup and + * enable IPIs as APs come online. + */ + if (irq <= GIC_LAST_SGI) + gic_v3_enable_intr_periph(&pargs); + else + smp_rendezvous(NULL, gic_v3_enable_intr_periph, NULL, + &pargs); + return; + } + + sc = device_get_softc(dev); + + if (irq >= GIC_FIRST_SPI && irq <= GIC_LAST_SPI) { /* SPIs in distributor */ gic_d_write(sc, 4, GICD_ISENABLER(irq), GICD_I_MASK(irq)); gic_v3_wait_for_rwp(sc, DIST); @@ -887,8 +942,6 @@ gic_v3_bind_intr(device_t dev, struct intr_irqsrc *isrc) int cpu; gi = (struct gic_v3_irqsrc *)isrc; - if (gi->gi_irq <= GIC_LAST_PPI) - return (EINVAL); KASSERT(gi->gi_irq >= GIC_FIRST_SPI && gi->gi_irq <= GIC_LAST_SPI, ("%s: Attempting to bind an invalid IRQ", __func__)); @@ -916,6 +969,7 @@ gic_v3_bind_intr(device_t dev, struct intr_irqsrc *isrc) static void gic_v3_init_secondary(device_t dev) { + struct gic_v3_setup_periph_args pargs; device_t child; struct gic_v3_softc *sc; gic_v3_initseq_t *init_func; @@ -937,18 +991,25 @@ gic_v3_init_secondary(device_t dev) } } + pargs.dev = dev; + /* Unmask attached SGI interrupts. */ for (irq = GIC_FIRST_SGI; irq <= GIC_LAST_SGI; irq++) { isrc = GIC_INTR_ISRC(sc, irq); - if (intr_isrc_init_on_cpu(isrc, cpu)) - gic_v3_enable_intr(dev, isrc); + if (intr_isrc_init_on_cpu(isrc, cpu)) { + pargs.isrc = isrc; + gic_v3_enable_intr_periph(&pargs); + } } /* Unmask attached PPI interrupts. */ for (irq = GIC_FIRST_PPI; irq <= GIC_LAST_PPI; irq++) { isrc = GIC_INTR_ISRC(sc, irq); - if (intr_isrc_init_on_cpu(isrc, cpu)) - gic_v3_enable_intr(dev, isrc); + if (intr_isrc_init_on_cpu(isrc, cpu)) { + pargs.isrc = isrc; + gic_v3_setup_intr_periph(&pargs); + gic_v3_enable_intr_periph(&pargs); + } } for (i = 0; i < sc->gic_nchildren; i++) {