PCI interrupts passing DMA

[Aaro Koskinen]

Hello,

> I was toying with a programmable PCI card and wrote some code
> which DMAs a small block of data to the host, and then interrupts the
> host. The host checks the end of the block, and sees if it gets the
> value it expects.

> On an SMP P4 (hyperthreaded, with ServerWorks chipset) FreeBSD 4.8 UP,
> and on Linux 2.4.18, there is a huge delay between the interrupt being
> handled, and the DMA finally completing (from the host's perspective).
> Time enough for the interrupt handler to be triggered 3 or 4 times,
> and to print "foo" to a serial console line each time it notices
> that the DMA has not completed.
>
> The interesting thing is that on FreeBSD 4.8SMP, and FreeBSD
> 5.1-current (SMP), the data has arrived by the time the interrupt
> handler is called.
[...]
> My question is: What the heck could the SMP kernel be doing which
> causes the DMA to "complete" faster?

The chipset probably uses PCI bus (MSI-like mechanism) to deliver the
interrupt from the IO APIC to the local APIC, which means that the PCI
bridge(s) must complete the DMA transfer before the interrupt is
delivered to preserve the write order.

In PIC mode, the interrupt is delivered by the wire and it has no
effect on pending writes. A common solution is that the interrupt
handler must perform a read from the device to the force flushing of
buffers.

A.

-- 
Aaro Koskinen
E-mail: aaro at iki.fi            "I'm the ocean, I'm the giant undertow."
http://www.iki.fi/aaro