ioapic_assign_cpu() on active level-triggered interrupt

[Alexander Motin]

Hi.

I am working on driver for HPET event timers. It works mostly fine,
except after some cases when ioapic_assign_cpu() called while timer is
active. Under interrupt rate of 10KHz it is enough a dozen cpuset runs
to break it (with 1KHz - few dozens). When it happens, I can see that
timer is still running, interrupt status register is changing, but no
interrupts received.

Timer uses level-triggered interrupts, so it is tolerant to interrupt
losses. I have tried to not acknowledge some, and they have immediately
got back to me again, as expected for level-triggering. Timer runs in
periodic mode, so it doesn't need handling to continue counting.

I have reproduced it on two different i386 SMP systems: Core2Duo+ICH10
and Core i5+PCH. With more experiments I have found that I can't trigger
this issue if following patch applied:

--- io_apic.c.prev      2010-06-02 10:55:56.000000000 +0300
+++ io_apic.c   2010-06-04 17:45:51.000000000 +0300
@@ -363,7 +366,10 @@ ioapic_assign_cpu(struct intsrc *isrc, u
                printf(") to lapic %u vector %u\n", intpin->io_cpu,
                    intpin->io_vector);
        }
+       ioapic_disable_source(isrc, PIC_NO_EOI);
+       DELAY(10);
        ioapic_program_intpin(intpin);
+       ioapic_enable_source(isrc);
        /*
         * Free the old vector after the new one is established.  This
is done
         * to prevent races where we could miss an interrupt.

It is is almost a hack and 10us is completely experimental. But it looks
like changing interrupt's APIC and vector in some moments of interrupt
processing may be not a good idea.

Can somebody explain this behavior and propose some solution? Have
somebody seen it for regular PCI devices?

-- 
Alexander Motin