interesting(?) data on network interrupt servicing

[M. Warner Losh]

In message: <44220684.80300 at samsco.org>
            Scott Long <scottl at samsco.org> writes:
: Luigi Rizzo wrote:
: 
: > On Wed, Mar 22, 2006 at 10:18:44PM +0100, Poul-Henning Kamp wrote:
: > 
: >>In message <20060322131448.A42341 at xorpc.icir.org>, Luigi Rizzo writes:
: >>
: >>
: >>>>>   if (thread)
: >>>>>	isrc->is_pic->pic_disable_source(isrc, PIC_EOI);
: >>>>>
: >>>>>I have no idea, though, why the other pic_disable_source()
: >>>>>is so expensive. The 4-5k TSC ticks are approx 3us
: >>>>>
: >>>>>Any clues ?
: >>>>
: >>>>ISA bus access.
: >>>
: >>>yeah but this is a modern laptop with an apic,
: >>>not an 8259... i don't think it has any ISA bus unless there is
: >>>some strange emulation going on ?
: >>
: >>There is, how else would it boot MS-DOS ?
: > 
: > 
: > so anything we can do in the kernel config to remove that ?
: > (i don't have here the config Paolo is using...)
: > 
: 
: I've done extensive TSC measurements in here too about 18 months ago
: (see the commit logs for an optimization that I put in).  Part of the
: expense is the indirect function call, and part of it is the memory
: read.  I'm not sure if I completely agree with John that the read is
: over PCI, I think that for most cases it'll be on the front side bus.
: But in any case, it's an uncached read, so it's expensive.  I like
: John's patch for getting rid of it.  Beyond that, the other expenses
: come from using the icu_lock spinlock, and John and I think that there
: might be ways to reduce that.  INTR_FAST handlers bypass a lot of this
: too.  The only thing left after all of that is the bintime calls and
: the indirect function calls to the PIC/APIC code.  Those are there as
: part of the PIC abstraction, and there is really no way around them.
: I'd really like to see FreeBSD be able to get from int assertion to
: driver interrupt handler in 1000 ticks or less (at least for INTR_FAST),
: so any suggestions are welcome.

I've done measurements with a scope and DIO lines of interrupt
latency for INTR_FAST interrupt handlers.  I found that 5.3 performed
horribly (rarely could I hit a 20us window on a 400MHz celeron).  6.1
so far seems to be much better, but I've not made enough measurements
to know for sure.  In 5.3 we were deifnitely about 10x-20x worse than 1000
ticks (== CPU Clock cycles?) to a fast ISR.

Warner