interesting(?) data on network interrupt servicing

[John Baldwin]

On Wednesday 22 March 2006 15:41, Poul-Henning Kamp wrote:
> In message <20060322122906.A41691 at xorpc.icir.org>, Luigi Rizzo writes:
> 
> >    if (thread)
> >	isrc->is_pic->pic_disable_source(isrc, PIC_EOI);
> >
> >I have no idea, though, why the other pic_disable_source()
> >is so expensive. The 4-5k TSC ticks are approx 3us
> >
> >Any clues ?
> 
> ISA bus access.

Not for APIC.  Still, the I/O APIC for modern systems lives out on
the PCI bus, so you are talking about PCI bus access.  Here's one
patch you can try (I've not got benchmarks to show if it helps at all).
It cuts this function down from a PCI read and then a PCI write to
just a PCI write:

--- //depot/vendor/freebsd/src/sys/i386/i386/io_apic.c	2005/11/16 20:30:33
+++ //depot/user/jhb/acpipci/i386/i386/io_apic.c	2005/11/21 19:21:59
@@ -90,6 +90,7 @@
 	u_int io_masked:1;
 	int io_dest:5;
 	int io_bus:4;
+	uint32_t io_lowreg;
 };
 
 struct ioapic {
@@ -208,9 +209,7 @@
 
 	mtx_lock_spin(&icu_lock);
 	if (intpin->io_masked) {
-		flags = ioapic_read(io->io_addr,
-		    IOAPIC_REDTBL_LO(intpin->io_intpin));
-		flags &= ~(IOART_INTMASK);
+		flags = intpin->io_lowreg & ~IOART_INTMASK;
 		ioapic_write(io->io_addr, IOAPIC_REDTBL_LO(intpin->io_intpin),
 		    flags);
 		intpin->io_masked = 0;
@@ -227,9 +226,7 @@
 
 	mtx_lock_spin(&icu_lock);
 	if (!intpin->io_masked && !intpin->io_edgetrigger) {
-		flags = ioapic_read(io->io_addr,
-		    IOAPIC_REDTBL_LO(intpin->io_intpin));
-		flags |= IOART_INTMSET;
+		flags = intpin->io_lowreg | IOART_INTMSET;
 		ioapic_write(io->io_addr, IOAPIC_REDTBL_LO(intpin->io_intpin),
 		    flags);
 		intpin->io_masked = 1;
@@ -320,6 +317,7 @@
 
 	/* Write the values to the APIC. */
 	mtx_lock_spin(&icu_lock);
+	intpin->io_lowreg = low;
 	ioapic_write(io->io_addr, IOAPIC_REDTBL_LO(intpin->io_intpin), low);
 	value = ioapic_read(io->io_addr, IOAPIC_REDTBL_HI(intpin->io_intpin));
 	value &= ~IOART_DEST;

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org