Arm64 stack issues (was Re: FreeBSD status for/on ODroid-C2?)
Mark Millard
markmigm at gmail.com
Thu Feb 2 08:37:59 UTC 2017
I'm out of my element here but I will note one difference
between what I read in the likes of:
http://infocenter.arm.com/help/topic/com.arm.doc.den0024a/DEN0024A_v8_architecture_PG.pdf
and what I see in /usr/src/sys/arm/arm/gic.c 's arm_gic_intr .
The written description says (quoting 10.6 The Generic Interrupt
Controller and 10.6.3 Interrupt Handling):
> Interrupts can either be edge-triggered (considered to be asserted when the GIC detects a rising edge on the relevant input, and to remain asserted until cleared) or level-sensitive (considered to be asserted only when the relevant input to the GIC is HIGH).
>
> . . .
>
> The priority and list of cores to which an interrupt can be delivered to are all configured in the Distributor. An interrupt asserted to the Distributor by a peripheral is in the Pending state (or Active and Pending if it was already Active). The Distributor determines the highest priority pending interrupt that can be delivered to a core and forwards that to the CPU interface of the core. At the CPU interface, the interrupt is in turn signaled to the core, at which point the core takes the FIQ or IRQ exception.
>
> The core executes the exception handler in response. The handler must query the interrupt ID from a CPU interface register and begin servicing the interrupt source. When finished, the handler must write to a CPU interface register to report the end of processing.
>
> • For a given interrupt the typical sequence is:
>
> • Inactive -> Pending
> When the interrupt is asserted by the peripheral.
>
> • Pending -> Active
> When the handler acknowledges the interrupt.
>
> • Active -> Inactive
> When the handle[r] has finished dealing with the interrupt.
>
> . . .
>
> The top-level interrupt handler reads the Interrupt Acknowledge Register from the CPU Interface block to obtain the interrupt ID.
>
> As well as returning the interrupt ID, the read causes the interrupt to be marked as active in the Distributor. Once the interrupt ID is known (identifying the interrupt source), the top-level handler can now dispatch a device-specific handler to service the interrupt.
>
> When the device-specific handler finishes execution, the top-level handler writes the same interrupt ID to the End of Interrupt (EoI) register in the CPU Interface block, indicating the end of interrupt processing.
So that wording indicates that the write to GICC_EOIR should
be after the dispatched activity (after "servicing"), not
before. I did not find anything indicating that edge-triggered
vs. level triggered would be different for this, for example.
(But being unfamiliar I could have missed something.)
In two cases below the code has the write to the GICC_EOIR
before the dispatch (so before the servicing activity),
possibly allowing another interrupt during or even before
the dispatched activity (say if the state for the irq is
active-and-pending at the time of the GICC_EOIR write or
if there is a lower priority interrupt pending at that
time):
> if (irq <= GIC_LAST_SGI) {
> #ifdef SMP
> /* Call EOI for all IPI before dispatch. */
> gic_c_write_4(sc, GICC_EOIR, irq_active_reg);
> intr_ipi_dispatch(sgi_to_ipi[gi->gi_irq], tf);
> goto next_irq;
> #else
> . . .
> #endif
> }
>
> . . .
> if ((gi->gi_flags & GI_FLAG_EARLY_EOI) == GI_FLAG_EARLY_EOI)
> gic_c_write_4(sc, GICC_EOIR, irq_active_reg);
>
> if (intr_isrc_dispatch(&gi->gi_isrc, tf) != 0) {
> gic_irq_mask(sc, irq);
> if ((gi->gi_flags & GI_FLAG_EARLY_EOI) != GI_FLAG_EARLY_EOI)
> gic_c_write_4(sc, GICC_EOIR, irq_active_reg);
> device_printf(sc->gic_dev, "Stray irq %u disabled\n", irq);
> }
>
> next_irq:
. . .
Note: GI_FLAG_EARLY_EOI was set for edge triggered:
> /* For MSI/MSI-X we should have already configured these */
> if ((gi->gi_flags & GI_FLAG_MSI) == 0) {
> if (pol == INTR_POLARITY_CONFORM)
> pol = INTR_POLARITY_LOW; /* just pick some */
> if (trig == INTR_TRIGGER_CONFORM)
> trig = INTR_TRIGGER_EDGE; /* just pick some */
>
> gi->gi_pol = pol;
> gi->gi_trig = trig;
>
> /* Edge triggered interrupts need an early EOI sent */
> if (gi->gi_pol == INTR_TRIGGER_EDGE)
> gi->gi_flags |= GI_FLAG_EARLY_EOI;
> }
===
Mark Millard
markmi at dsl-only.net
On 2017-Feb-1, at 7:07 PM, Mark Millard <markmi at dsl-only.net> wrote:
> I temporarily modified the Spurious-interrupt-detected notice to also report
> irq and sc->nirqs :
>
>
> . . .
> #define gic_c_read_4(_sc, _reg) \
> bus_space_read_4((_sc)->gic_c_bst, (_sc)->gic_c_bsh, (_reg))
>
> . . .
> int
> arm_gic_intr(void *arg)
> {
> struct arm_gic_softc *sc = arg;
> struct gic_irqsrc *gi;
> uint32_t irq_active_reg, irq;
> struct trapframe *tf;
>
> irq_active_reg = gic_c_read_4(sc, GICC_IAR);
> irq = irq_active_reg & 0x3FF;
>
> /*
> . . .
> */
>
> if (irq >= sc->nirqs) {
> #ifdef GIC_DEBUG_SPURIOUS
> device_printf(sc->gic_dev,
> "Spurious interrupt %d detected of %d: last irq: %d on CPU%d\n",
> irq, sc->nirqs,
> sc->last_irq[PCPU_GET(cpuid)], PCPU_GET(cpuid));
> #endif
> return (FILTER_HANDLED);
> }
> . . .
>
>
> The result was irq==1023 and sc->nirqs==224 in every message
> that I've seen so far. 1023==0x3FF .
>
> Looking around I found in:
>
> http://www.cl.cam.ac.uk/research/srg/han/ACS-P35/zynq/arm_gic_architecture_specification.pdf
>
> the following on various reasons why 1023 would show up (quoting):
>
>
>
> • A processor reads the GICC_IAR and obtains the interrupt ID 1023, indicating a spurious interrupt. The processor can return from its interrupt service routine without writing to its GICC_EOIR.
>
> The spurious interrupt ID indicates that the original interrupt is no longer pending, typically because another target processor is handling it.
>
> . . .
>
> The GIC architecture reserves interrupt ID numbers 1020-1023 for special purposes. In a GICv1 implementation that does not implement the GIC Security Extensions, the only one of these used is ID 1023. This value is returned to a processor, in response to an interrupt acknowledge, if there is no pending interrupt with sufficient priority for it to be signaled to the processor. It is described as a response to a spurious interrupt.
>
> Note
>
> A race condition can cause a spurious interrupt. For example, a spurious interrupt can occur if a processor writes a 1 to a field in an GICD_ICENABLERn that corresponds to a pending interrupt after the CPU interface has signaled the interrupt to the processor and the processor has recognized the interrupt, but before the processor has read from the GICC_IAR.
>
> . . .
>
> • If a read of the GICC_IAR does not match the security of the interrupt, the GICC_IAR read does not acknowledge any interrupt and returns the value:
>
> • 1022 for a Secure read when the highest priority interrupt is Non-secure
>
> • 1023 for a Non-secure read when the highest priority interrupt is Secure.
> . . .
>
> A read of the GICC_IAR returns the interrupt ID of the highest priority pending interrupt for the CPU interface. The read returns a spurious interrupt ID of 1023 if any of the following apply:
>
> • forwarding of interrupts by the Distributor to the CPU interface is disabled
>
> • signaling of interrupts by the CPU interface to the connected processor is disabled
>
> • no pending interrupt on the CPU interface has sufficient priority for the interface to signal it to the processor.
>
>
> • The following sequence of events is an example of when the GIC returns an interrupt ID of 1023, and shows how reads of the GICC_IAR can be timing critical:
>
> 1. A peripheral asserts a level-sensitive interrupt.
>
> 2. The interrupt has sufficient priority and therefore the GIC signals it to a targeted processor.
>
> 3. The peripheral deasserts the interrupt. Because there is no other pending interrupt of sufficient priority, the GIC deasserts the interrupt request to the processor.
>
> 4. Before it has recognized the deassertion of the interrupt request from stage 3, the targeted processor reads the GICC_IAR. Because there is no interrupt with sufficient priority to signal to the processor, the GIC returns the spurious ID value of 1023.
>
>
> The determination of the returned interrupt ID is more complex if the GIC supports interrupt grouping
>
> . . .
>
>
> Interrupt signaling of the required interrupt group by CPU interface disabled
>
>
>
> ===
> Mark Millard
> markmi at dsl-only.net
_______________________________________________
freebsd-arm at freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org"
More information about the freebsd-arm
mailing list