dummynet, em driver, device polling issues :-((

[Ferdinand Goldmann]

Hello All!

We are using a 1.2GHz xSeries machine to do poor man's traffic shaping for the 
student hostels connected to our campus. We are using dummynet to do shaping, 
there is no stateful filtering, there are however several filter rules to lock 
out people and to do MAC based filtering. Number of users is ranging from 
several hundred up to 2-3000. We are running a recent 5.4-STABLE.

The machine is fitted with two onboard Intel 100Mbit interfaces. One of the two 
was running on very high load (the internal interface acting as gateway for all 
of the hostels, where no shaping is done), so we put in a spare Intel Gbit card:

em0 at pci1:5:0:   class=0x020000 card=0x10028086 chip=0x10268086 rev=0x04 hdr=0x00
     vendor   = 'Intel Corporation'
     device   = '82545GM Gigabit Ethernet Controller'

Initial observations showed that the system had a very high interrupt load, 
most of the times > 50%, and lagged a little bit. Plus, performance-testing 
with tcpspray yielded bad results at around 10MB/s, or even worse. So we tried 
enabling device polling on the em0 interface. After some testing, we set 
HZ=1000 and kern.polling.burst_max: 300. This made things slightly better, our 
tcpspray tests now reaching a ~13MB/s throughput.

However, now the input errors on the em0 interface were rapidly rising:
Name    Mtu Network       Address              Ipkts Ierrs    Opkts Oerrs  Coll
em0    1500 <Link#3>      00:0e:0c:6d:4c:f9 3738461290 8605788 3613603979     0 
     0

So for now, the polling is disabled again.

On the mailing list, I have read about performance problems with the em driver, 
and problems with device polling as well.

* are there any magic performance tips for use with the em driver? Any way to 
eliminate the input errors while using device polling? Or is all of this code 
too broken at the moment?

* is there a driver by Intel which will work well? Has anyone tested such a 
driver with a network-performance hungry application?

For now, the machine seems to run stable, no more errors with device polling 
disabled. But performance is far lower than it should be. :-(

Kind regards and thanks for any input!
-- 
 >> Ferdinand Goldmann                                   ////  |          |
 >> EMail:  Ferdinand.Goldmann at zid.uni-linz.ac.at       |--00  |    UNIX  |
 >> Tel. : +43/732/2468/9398 Fax. : +43/732/2468/9397   C   ^  |          |
 >> EMail:  Ferdinand.Goldmann at zid.uni-linz.ac.at        \ ~/  ~~~|~~~~~~~~
 >> PGP D4CF 8AA4 4B2A 7B88 65CA  5EDC 0A9B FA9A 13EA B993| |-----3