From nobody Mon May 22 15:24:22 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QQ1T32HNhz4BZyF; Mon, 22 May 2023 15:24:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QQ1T30dlDz3N3j; Mon, 22 May 2023 15:24:23 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684769063; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tixsmHVS9MPqS+zdHlx1eVtf4jrTOi9ShsQMaam746I=; b=DkdaE/lb74MzxV7zknBIZm/Dxc8BcdeGqeZbW8H30g1HmbcemVRkQTWAlX8zCh4jh2OchM BE5vTczZwVzpStiVEJ3tMVPp0sU1UYLWDpOcXgXcZygMiXkvn3qeMZ1upb38B0XfFEn6tL Q9lVvEfokiLKV0ih2KZF0wPxmKCxqJCRr0E5gFvERppmoIMiB75W06TqkorODkiGRydBp3 hKLkP/m9oXuV193fpD83u57roHqB24DowToM7qQ8dV3DauTc7vwp+Yz98EEH2PM0yZYSUo FCVoaylzwH8bSGQqss8xpAE5r3D3tDEuMrs+rZbK1SZWliQ7GAT4P2mUUogs0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684769063; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tixsmHVS9MPqS+zdHlx1eVtf4jrTOi9ShsQMaam746I=; b=iumnoXDtTDHaEEr7D6OKj47jObAU5+d6x/5eNx/xIzOwc4w6gsIQdMOZDj7tbP/P22J3nm UEAef0H6EakTAKp0KDyiFUObt8c1kNBm4LrDuOse4Gkpi/pH/0X0wTonNeHv2mO4MJfPLy QkzeOoTjZ1YlNphyPev4XQUZBwT2vVTpI4q6PMxZdJUnZ4Aj4/d+ic58q8BcyIElJX1LIr 2QZ3agE6wXwE/ATfSi7ZqMYSYt7VK1JcczuIhE5LFsDtIbt1gkZH5rme017skIuUdS7O3z EOz/r27+ApNmjq6qwyz2bi9+a3pXVtYEAuRCGin6GDmIMV8qQu3zLR9x15M6vg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1684769063; a=rsa-sha256; cv=none; b=XzMja2OIUFMvvDp2FY3YCXuFBIM6PoTzOoDuWaxa97p6oar/YWy11tLtvhuktWil0GswfN Ifbcmxnsivv6LJeN3n2CzjKwxlpbR/ZfWk/N060IeTs2zuSZijspsZJrj/DYB1rbmiS1+J knuv27hIsyZ4xB+h5JlJ5X5MQqRosr3qzUax1pgS5LWBeAwQh9qxybWFrgTQWDxajOFKtb Kqg/LuXH5DfFhQppoxHB6UTvQVxCWs1esqstcJmH3oGmFK7tvsaXmO1uG8rKW/nWlB0CIa 0iCss6mZTXdFzOfV6GYPArAPqlScUuhbs+p552IqVmUF2ahADfsokYfAlAKlbA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QQ1T26pB7zNSJ; Mon, 22 May 2023 15:24:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 34MFOM46041314; Mon, 22 May 2023 15:24:22 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 34MFOMNi041313; Mon, 22 May 2023 15:24:22 GMT (envelope-from git) Date: Mon, 22 May 2023 15:24:22 GMT Message-Id: <202305221524.34MFOMNi041313@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kyle Evans Subject: git: 172af24449cd - main - arm64: gicv3: setup PPIs on all APs after they're online List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 172af24449cd8d34339172d125832b7ecd274213 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=172af24449cd8d34339172d125832b7ecd274213 commit 172af24449cd8d34339172d125832b7ecd274213 Author: Kyle Evans AuthorDate: 2023-05-15 17:21:45 +0000 Commit: Kyle Evans CommitDate: 2023-05-22 15:23:53 +0000 arm64: gicv3: setup PPIs on all APs after they're online For all PPIs setup earlier than SI_SUB_SMP, PIC_INIT_SECONDARY ends up cleaning these up for each AP as it comes online. Once they're online, we don't currently do anything to make sure they're configured for other APs. Fix it by using smp_rendezvous for the meaty bits of configuring a PPI, which will just do single-thread behavior before APs are online but do the right thing for other CPUs after. While we're here, make sure redistributor config is correct for other APs as they come online in gic_v3_init_secondary. Reported/Tested by: Souradeep Chakrabarti (Microsoft/Hyper-V) Reviewed by: andrew (before slight refactor) Differential Revision: https://reviews.freebsd.org/D40112 --- sys/arm64/arm64/gic_v3.c | 157 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 109 insertions(+), 48 deletions(-) diff --git a/sys/arm64/arm64/gic_v3.c b/sys/arm64/arm64/gic_v3.c index dfa0f6c7cad1..e9754797d095 100644 --- a/sys/arm64/arm64/gic_v3.c +++ b/sys/arm64/arm64/gic_v3.c @@ -835,15 +835,66 @@ gic_v3_map_intr(device_t dev, struct intr_map_data *data, return (error); } +struct gic_v3_setup_periph_args { + device_t dev; + struct intr_irqsrc *isrc; +}; + +static void +gic_v3_setup_intr_periph(void *argp) +{ + struct gic_v3_setup_periph_args *args = argp; + struct intr_irqsrc *isrc = args->isrc; + struct gic_v3_irqsrc *gi = (struct gic_v3_irqsrc *)isrc; + device_t dev = args->dev; + u_int irq = gi->gi_irq; + struct gic_v3_softc *sc = device_get_softc(dev); + uint32_t reg; + + MPASS(irq <= GIC_LAST_SPI); + + /* + * We need the lock for both SGIs and PPIs for an atomic CPU_SET() at a + * minimum, but we also need it below for SPIs. + */ + mtx_lock_spin(&sc->gic_mtx); + + if (isrc->isrc_flags & INTR_ISRCF_PPI) + CPU_SET(PCPU_GET(cpuid), &isrc->isrc_cpu); + + if (irq >= GIC_FIRST_PPI && irq <= GIC_LAST_SPI) { + /* Set the trigger and polarity */ + if (irq <= GIC_LAST_PPI) + reg = gic_r_read(sc, 4, + GICR_SGI_BASE_SIZE + GICD_ICFGR(irq)); + else + reg = gic_d_read(sc, 4, GICD_ICFGR(irq)); + if (gi->gi_trig == INTR_TRIGGER_LEVEL) + reg &= ~(2 << ((irq % 16) * 2)); + else + reg |= 2 << ((irq % 16) * 2); + + if (irq <= GIC_LAST_PPI) { + gic_r_write(sc, 4, + GICR_SGI_BASE_SIZE + GICD_ICFGR(irq), reg); + gic_v3_wait_for_rwp(sc, REDIST); + } else { + gic_d_write(sc, 4, GICD_ICFGR(irq), reg); + gic_v3_wait_for_rwp(sc, DIST); + } + } + + mtx_unlock_spin(&sc->gic_mtx); +} + static int gic_v3_setup_intr(device_t dev, struct intr_irqsrc *isrc, struct resource *res, struct intr_map_data *data) { - struct gic_v3_softc *sc = device_get_softc(dev); struct gic_v3_irqsrc *gi = (struct gic_v3_irqsrc *)isrc; + struct gic_v3_setup_periph_args pargs; enum intr_trigger trig; enum intr_polarity pol; - uint32_t reg; u_int irq; int error; @@ -872,41 +923,18 @@ gic_v3_setup_intr(device_t dev, struct intr_irqsrc *isrc, gi->gi_trig = trig; } - /* - * XXX - In case that per CPU interrupt is going to be enabled in time - * when SMP is already started, we need some IPI call which - * enables it on others CPUs. Further, it's more complicated as - * pic_enable_source() and pic_disable_source() should act on - * per CPU basis only. Thus, it should be solved here somehow. - */ - if (isrc->isrc_flags & INTR_ISRCF_PPI) - CPU_SET(PCPU_GET(cpuid), &isrc->isrc_cpu); - - if (irq >= GIC_FIRST_PPI && irq <= GIC_LAST_SPI) { - mtx_lock_spin(&sc->gic_mtx); - - /* Set the trigger and polarity */ - if (irq <= GIC_LAST_PPI) - reg = gic_r_read(sc, 4, - GICR_SGI_BASE_SIZE + GICD_ICFGR(irq)); - else - reg = gic_d_read(sc, 4, GICD_ICFGR(irq)); - if (trig == INTR_TRIGGER_LEVEL) - reg &= ~(2 << ((irq % 16) * 2)); - else - reg |= 2 << ((irq % 16) * 2); - - if (irq <= GIC_LAST_PPI) { - gic_r_write(sc, 4, - GICR_SGI_BASE_SIZE + GICD_ICFGR(irq), reg); - gic_v3_wait_for_rwp(sc, REDIST); - } else { - gic_d_write(sc, 4, GICD_ICFGR(irq), reg); - gic_v3_wait_for_rwp(sc, DIST); - } - - mtx_unlock_spin(&sc->gic_mtx); + pargs.dev = dev; + pargs.isrc = isrc; + if (isrc->isrc_flags & INTR_ISRCF_PPI) { + /* + * If APs haven't been fired up yet, smp_rendezvous() will just + * execute it on the single CPU and gic_v3_init_secondary() will + * clean up afterwards. + */ + smp_rendezvous(NULL, gic_v3_setup_intr_periph, NULL, &pargs); + } else if (irq >= GIC_FIRST_SPI && irq <= GIC_LAST_SPI) { + gic_v3_setup_intr_periph(&pargs); gic_v3_bind_intr(dev, isrc); } @@ -951,23 +979,50 @@ gic_v3_disable_intr(device_t dev, struct intr_irqsrc *isrc) panic("%s: Unsupported IRQ %u", __func__, irq); } +static void +gic_v3_enable_intr_periph(void *argp) +{ + struct gic_v3_setup_periph_args *args = argp; + struct gic_v3_irqsrc *gi = (struct gic_v3_irqsrc *)args->isrc; + device_t dev = args->dev; + struct gic_v3_softc *sc = device_get_softc(dev); + u_int irq = gi->gi_irq; + + /* SGIs and PPIs in corresponding Re-Distributor */ + gic_r_write(sc, 4, GICR_SGI_BASE_SIZE + GICD_ISENABLER(irq), + GICD_I_MASK(irq)); + gic_v3_wait_for_rwp(sc, REDIST); +} + static void gic_v3_enable_intr(device_t dev, struct intr_irqsrc *isrc) { + struct gic_v3_setup_periph_args pargs; struct gic_v3_softc *sc; struct gic_v3_irqsrc *gi; u_int irq; - sc = device_get_softc(dev); gi = (struct gic_v3_irqsrc *)isrc; irq = gi->gi_irq; + pargs.isrc = isrc; + pargs.dev = dev; if (irq <= GIC_LAST_PPI) { - /* SGIs and PPIs in corresponding Re-Distributor */ - gic_r_write(sc, 4, GICR_SGI_BASE_SIZE + GICD_ISENABLER(irq), - GICD_I_MASK(irq)); - gic_v3_wait_for_rwp(sc, REDIST); - } else if (irq >= GIC_FIRST_SPI && irq <= GIC_LAST_SPI) { + /* + * SGIs only need configured on the current AP. We'll setup and + * enable IPIs as APs come online. + */ + if (irq <= GIC_LAST_SGI) + gic_v3_enable_intr_periph(&pargs); + else + smp_rendezvous(NULL, gic_v3_enable_intr_periph, NULL, + &pargs); + return; + } + + sc = device_get_softc(dev); + + if (irq >= GIC_FIRST_SPI && irq <= GIC_LAST_SPI) { /* SPIs in distributor */ gic_d_write(sc, 4, GICD_ISENABLER(irq), GICD_I_MASK(irq)); gic_v3_wait_for_rwp(sc, DIST); @@ -1010,8 +1065,6 @@ gic_v3_bind_intr(device_t dev, struct intr_irqsrc *isrc) int cpu; gi = (struct gic_v3_irqsrc *)isrc; - if (gi->gi_irq <= GIC_LAST_PPI) - return (EINVAL); KASSERT(gi->gi_irq >= GIC_FIRST_SPI && gi->gi_irq <= GIC_LAST_SPI, ("%s: Attempting to bind an invalid IRQ", __func__)); @@ -1039,6 +1092,7 @@ gic_v3_bind_intr(device_t dev, struct intr_irqsrc *isrc) static void gic_v3_init_secondary(device_t dev) { + struct gic_v3_setup_periph_args pargs; device_t child; struct gic_v3_softc *sc; gic_v3_initseq_t *init_func; @@ -1060,18 +1114,25 @@ gic_v3_init_secondary(device_t dev) } } + pargs.dev = dev; + /* Unmask attached SGI interrupts. */ for (irq = GIC_FIRST_SGI; irq <= GIC_LAST_SGI; irq++) { isrc = GIC_INTR_ISRC(sc, irq); - if (intr_isrc_init_on_cpu(isrc, cpu)) - gic_v3_enable_intr(dev, isrc); + if (intr_isrc_init_on_cpu(isrc, cpu)) { + pargs.isrc = isrc; + gic_v3_enable_intr_periph(&pargs); + } } /* Unmask attached PPI interrupts. */ for (irq = GIC_FIRST_PPI; irq <= GIC_LAST_PPI; irq++) { isrc = GIC_INTR_ISRC(sc, irq); - if (intr_isrc_init_on_cpu(isrc, cpu)) - gic_v3_enable_intr(dev, isrc); + if (intr_isrc_init_on_cpu(isrc, cpu)) { + pargs.isrc = isrc; + gic_v3_setup_intr_periph(&pargs); + gic_v3_enable_intr_periph(&pargs); + } } for (i = 0; i < sc->gic_nchildren; i++) {