dummynet, em driver, device polling issues :-((

[Kevin Day]

On Oct 4, 2005, at 6:56 PM, Dave+Seddon wrote:


> You mention your running at "near" line rate.  What are you pushing  
> or pulling?  Whats the rough spec of these machines pushing out  
> this much data? What setting do you have for the polling?  I've  
> been trying to do near line rate and can't even get close with new  
> HP-DL380s (Single 3.4 Ghz Xeon).  I think the PCI bus might be the  
> problem.  The em Intel NICs I found to be very slow and stop after  
> about 3 hours.  - The Intel NICs I have are dual port, although  
> they end up on seperate IRQs.
>

In one case, we had a system acting as a router. It was a Dell  
PowerEdge 2650, with two dual "server" adapters. each were on  
separate PCI busses. 3 were "lan" links, and one was a "wan" link.  
The lan links were receiving about 300mbps each, all going out the  
"wan" link at near 900mbps at peak. We were never able to get above  
944mbps, but I never cared enough to figure out where the bottleneck  
was there.

This was with PCI-X, and a pretty stripped config on the server side.

Nothing fancy on polling, i think we set HZ to 10000, turned on  
idle_poll, and set user_frac to 10 because we had some cpu hungry  
tasks that were not a high priority.

For anyone watching, the config we had there that we were successful  
with was:

em0 at pci2:6:0:   class=0x020000 card=0x10118086 chip=0x10108086  
rev=0x01 hdr=0x00
     vendor   = 'Intel Corporation'
     device   = '82546EB Dual Port Gigabit Ethernet Controller (Copper)'
     class    = network
     subclass = ethernet
em1 at pci2:6:1:   class=0x020000 card=0x10118086 chip=0x10108086  
rev=0x01 hdr=0x00
     vendor   = 'Intel Corporation'
     device   = '82546EB Dual Port Gigabit Ethernet Controller (Copper)'
     class    = network
     subclass = ethernet
em2 at pci1:8:0:   class=0x020000 card=0x10128086 chip=0x10128086  
rev=0x01 hdr=0x00
     vendor   = 'Intel Corporation'
     device   = '82546EB Dual Port Gigabit Ethernet Controller (Fiber)'
     class    = network
     subclass = ethernet
em3 at pci1:8:1:   class=0x020000 card=0x10128086 chip=0x10128086  
rev=0x01 hdr=0x00
     vendor   = 'Intel Corporation'
     device   = '82546EB Dual Port Gigabit Ethernet Controller (Fiber)'
     class    = network
     subclass = ethernet



We also have some web servers that are sending 300-400mbps each at  
peak using thttpd or lighttpd, with the built in Dell 2850 em parts.  
They also are connected via PCI-X speed buses internally.