Device polling performance
TM4525 at aol.com
TM4525 at aol.com
Mon Sep 27 12:07:35 PDT 2004
In a message dated 9/25/04 4:24:07 PM Eastern Daylight Time,
stheg_olloydson at yahoo.com writes:
>The EVIDENCE is to the contrary, since it seems that a 2.4Ghz system
>will be saturated when bridging ~250Kpps with device-polling enabled,
>based on polling stats and userland benchmarking, even though the
>system claims to be 100% idle. Interestingly, its about the same with
>The POINT is that since there is no way to measure the performance,
>you've got a bunch of guys who think they've figured something out
>touting device-polling without having a clue what the performance
>advantages (or consequences) are, so it might as well be black magic,
>or snake oil, since you are as blind as a bat in your assessments.
Please post your "polling stats and userland benchmarking" results. I
would be very interested seeing them as I was thinking of moving to
NICs that would benefit from polling. However, because you have
"EVIDENCE ... to the contrary", I may hold off. On the other hand, you
do go on to say "there is no way to measure the performance" and "you
are as blind as a bat in your assessments", so also please post your
test methodology. I need to make my decision on reliable, repeatable
Also, when you post, would you please wrap your lines to a shorter
length? Not everyone on the list uses AOL Reader, like you.
The "evidence" is a bit circumstantial in the absence of working tools, but
here are some "observations". There's also an assumption that the knobs
associated with polling work as expected.
Test machine is a 2.4Ghz celeron box with dual Intel NICs (em driver)
on a 32bit, 33Mhz bus, running FreeBSD 4.9. Now I realize that a 133Mhz,
64bit bus is 8x faster and you certainly wouldnt use these NICs on a real
network, but for the purpose of a control it doesnt matter, since both
tests are on the same MB.
RXdescriptors (receive ring size) = 512
(Note that the burst never exceeded 100 at any time)
I'm firing a controlled stream of 100K pps through the box (bridging). With
only normal userland (idle) usage, the box happily goes about its way.
Top shows 0-1.5% usage.
I started a cpu intensive userland task (buildworld or something of the sort).
The system started to lose packets with a user_frac setting of 78, which
implies that the system requires about 22% of the cpu to successfully
manage the task, assuming the knob works (it appears to). The same
machine, with interrupts enabled, uses about 26%, according to "top".
HOWEVER, setting hz back to 100, with interrupts enabled the usage
went down under 25%. Given that, it can be argued that there is less
than a 5% bonus for polling, which makes a lot more sense than what
some of the kooks have been saying.
Of course the point here wasn't to prove the difference, which Im still
not sure of, but the evidence certainly is that "top" doesn't properly
account for CPU usage in device_polling mode.. I'd expect a small
bonus, but nothing earth-shattering, as the machine still has to do
the same amount of work. Its not like the machine is really servicing
an interrupt for every packet, since controllers have hold offs so they
don't generate interrupts on top of each other, and multiple events
are regularly handled with a single interrupt.
Polling gives the appearance of a machine happily going about its
business no matter how much traffic you throw at it, but what happens
is that you lose packets when it becomes overmatched, which never
happens on a system with interrupts enabled before it goes into livelock.
While livelock isn't a good thing, if it only happens occasionally, at least
you aren't losing packets. Additionally, with a HZ setting of 1000 you're
also introducing quite a bit of latency:
additional_latency = up to 1ms in receive ring + transmit time for burst-1
Increasing HZ further would reduce the latency, but adds more overhead,
which slims the advantages and defeats the purpose of polling in the first
The bottom line is that there isn't so much difference as to think that
is going to save the day, but if you don't care about latency or losing
it can be useful in allocating cpu cycles to user space, if thats your
More information about the freebsd-questions