Device polling performance

[TM4525 at aol.com]

In a message dated 9/25/04 4:24:07 PM Eastern Daylight Time, 
stheg_olloydson at yahoo.com writes:
>The EVIDENCE is to the contrary, since it seems that a 2.4Ghz system
>will be saturated when bridging ~250Kpps with device-polling enabled,
>based on polling stats and userland benchmarking, even though the
>system claims to be 100% idle. Interestingly, its about the same with
>interrupt enabled.
>
>The POINT is that since there is no way to measure the performance,
>you've got a bunch of guys who think they've figured something out
>touting device-polling without having a clue what the performance
>advantages (or consequences) are, so it might as well be black magic,
>or snake oil, since you are as blind as a bat in your assessments.

Hello,

Please post your "polling stats and userland benchmarking" results. I
would be very interested seeing them as I was thinking of moving to
NICs that would benefit from polling. However, because you have
"EVIDENCE ... to the contrary", I may hold off. On the other hand, you
do go on to say "there is no way to measure the performance" and "you
are as blind as a bat in your assessments", so also please post your
test methodology. I need to make my decision on reliable, repeatable
facts.
Also, when you post, would you please wrap your lines to a shorter
length? Not everyone on the list uses AOL Reader, like you.
------------------------------------------------------------------------------
---------------------------
The "evidence" is a bit circumstantial in the absence of working tools, but 
here are some "observations". There's also an assumption that the knobs
associated with polling work as expected. 

Test machine is a 2.4Ghz celeron box with dual Intel NICs (em driver)
on a 32bit, 33Mhz bus, running FreeBSD 4.9. Now I realize that a 133Mhz,
64bit bus is 8x faster and you certainly wouldnt use these NICs on a real 
network, but for the purpose of a control it doesnt matter, since both 
tests are on the same MB.

Settings:

HZ=1000
each_burst=512
max_burst=1000
user_frac=variable

RXdescriptors (receive ring size) = 512

(Note that the burst never exceeded 100 at any time)

I'm firing a controlled stream of 100K pps through the box (bridging). With
only normal userland (idle) usage, the box happily goes about its way. 
Top shows 0-1.5% usage. 

I started a cpu intensive userland task (buildworld or something of the sort).
The system started to lose packets with a user_frac setting of 78, which
implies that the system requires about 22% of the cpu to successfully 
manage the task, assuming the knob works (it appears to). The same 
machine, with interrupts enabled, uses about 26%, according to "top". 
HOWEVER, setting hz back to 100, with interrupts enabled the usage 
went down under 25%. Given that, it can be argued that there is less 
than a 5% bonus for polling, which makes a lot more sense than what 
some of the kooks have been saying.

Of course the point here wasn't to prove the difference, which Im still 
not sure of, but the evidence certainly is that "top" doesn't properly 
account for CPU usage in device_polling mode.. I'd expect a small 
bonus, but nothing earth-shattering, as the machine still has to do 
the same amount of work. Its not like the machine is really servicing
an interrupt for every packet, since controllers have hold offs so they
don't generate interrupts on top of each other, and multiple events 
are regularly handled with a single interrupt.

Polling gives the appearance of a machine happily going about its 
business no matter how much traffic you throw at it, but what happens
is that you lose packets when it becomes overmatched, which never 
happens on a system with interrupts enabled before it goes into livelock. 
While livelock isn't a good thing, if it only happens occasionally, at least 
you aren't losing packets. Additionally, with a HZ setting of 1000 you're 
also introducing quite a bit of latency:

additional_latency = up to 1ms in receive ring + transmit time for burst-1
frames.

Increasing HZ further would reduce the latency, but adds more overhead, 
which slims the advantages and defeats the purpose of polling in the first 
place. 

The bottom line is that there isn't so much difference as to think that 
polling
is going to save the day, but if you don't care about latency or losing 
packets
it can be  useful in allocating cpu cycles to user space, if thats your 
priority.


TM