throughput and interrupts
Robert Watson
rwatson at FreeBSD.org
Tue Aug 15 12:02:06 UTC 2006
On Tue, 15 Aug 2006, Bachilo Dmitry wrote:
> I am completely stuck. I have a router, that rules three subnets. At first
> there were Allied Telesyn's netcards with realtek chipsets. I am using
> ipfw+natd and I had like 3,5 megabytes per second with 20 per cent of
> interrupt load. I had "discarded oversized frame" too often so I've replaced
> two main cards with 3Com 905 and now my xl0 and xl1 never discard frames.
> But I saw no speed boost and interrupts were also very high. I have
> discovered some IRQ conflicts and removed them, now I have single device per
> IRQ, but still i have near 20-30 % of interrupt load, when i download
> something through this router (and natd takes rest % of CPU). Some people
> advised me to use polling, so I've built kernel with HZ=1000, polling and
> tried it. Interrupts grew to 40-50% and speed decreased to 200-300 kb/s, so
> I had to turn the polling off.
>
> I just don't know what to try, to gain at least 9 mb/s, because this is what
> I get at home with the same cards, the same processor, with no polling and
> throgh the same provider.
I think it would be useful to compare the settled loads of the two systems, if
they are otherwise identical, using systat -vmstat 1 and top -S. I'd load the
systems up, leave the tools running, and wait a couple of minutes, then
compare snapshots of the output from both systems. top -S is particularly
informative, as it will tell you which ithreads are burning the most CPU, and
systat -vmstat will tell us the context switch rate and interrupt rates of
various sources.
There are some general optimization techniques that can be applied here -- for
example, if it's an SMP box, you might find that net.isr.direct=1 improves
performance. However, I think the real problem here is the performance
difference, which suggests a problem that needs resolving, rather than
optimization being required up front.
Thanks,
Robert N M Watson
Computer Laboratory
University of Cambridge
More information about the freebsd-current
mailing list