Intel 82574L Gigabit Ethernet Controller

[Shtorm]

On Sat, 2010-07-10 at 04:48 +1000, Ian Smith wrote:
> On Fri, 9 Jul 2010, Ryan Stone wrote:
> 
>  > No, defining EM_WATCHDOG as 10 * hz should mean that the watchdog
>  > expires after 10 seconds no matter what your kern.hz is.  hz is set to
>  > the number of ticks in a second.
> 
> Ok, one more probably wild punt .. Shtorm you say HZ=4000, giving:
> 
> ===
> And here is vmstat -i
> interrupt                          total       rate
> irq16: uhci0                        2673          2
> irq18: ehci0 uhci5                     2          0
> irq19: uhci2 uhci4                    70          0
> irq23: uhci3 ehci1                  8835          9
> cpu0: timer                      3743479       3957
> irq256: em0                       662219        700
> irq257: em0                       717531        758
> irq259: em0                        12142         12
> irq260: em0                         6100          6
> irq261: em1                       694169        733
> irq262: em1                       217388        229
> irq264: em1                       463023        489
> irq265: em1                           11          0
> cpu1: timer                      3743411       3957
> cpu6: timer                      3743408       3957
> cpu7: timer                      3743407       3957
> cpu4: timer                      3743408       3957
> cpu5: timer                      3743407       3957
> cpu3: timer                      3743408       3957
> cpu2: timer                      3743409       3957
> Total                           32731500      34599
> ===
> 
> a little more variant from 4000 than expected?  Originally:
> 
> ===
> but vmstat -i shows only one pair is used
> %vmstat -i
> interrupt                          total       rate
> irq1: atkbd0                         391          0
> irq18: ehci0 uhci5                     2          0
> irq19: uhci2 uhci4                    27          0
> irq23: uhci3 ehci1                  8463          0
> cpu0: timer                    107412943       7864
> irq256: em0                     93311951       6832
> irq257: em0                     90479067       6624
> irq260: em0                            2          0
> irq261: em1                     92966894       6806
> irq262: em1                     80298240       5879
> irq265: em1                            1          0
> cpu1: timer                    107412607       7864
> cpu2: timer                    107412785       7864
> cpu3: timer                    107412830       7864
> cpu5: timer                    107413210       7864
> cpu4: timer                    107413389       7864
> cpu7: timer                    107407337       7864
> cpu6: timer                    107413132       7864
> Total                         1216363271      89058
> ===
> 
> Was that with HZ=8000 ?  Or what?  In any case, em interrupt rates were 
> a whole lot higher then - but maybe it was just a lot busier on the net?
> 
> HZ=4000 ticks are 250ns, not 25ms.
> 
> Seems like you're not lacking horsepower .. unless you're using POLLING 
> (not indicated) or, say, dummynet pipes needing finer-grained output 
> queue scheduling, it might be worth trying the default HZ=1000 ?
> 
> Just curious along similar lines: what says sysctl kern.timecounter ?
> 
> cheers, Ian
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"

You are rigth, Ian, I'm using dumminet for traffic shaping, arount 2000
dynamic pipes (src and dst mask) with speed from 256 kbit up to 100
mbit. I've played with kern.hz a bit setting it to different values
between 1000 and 8000, 4000 gives best for my traffic load, two logs
taken with hz=4000 and hz=8000. And no, I'm not using polling, one of
the great features of intell network cards is interrupt moderation.
There is sysctls for this em driver:

%sysctl dev.em.0 | grep \[tr\]x
dev.em.0.rx_int_delay: 200
dev.em.0.tx_int_delay: 200
dev.em.0.rx_abs_int_delay: 500
dev.em.0.tx_abs_int_delay: 500

I've tried different values, more delay gives less context switches and
reduces cpu load in case of packet forwarding a lot. And you can see I'm
getting watchdog timeouts regardless of this settings, even tried
everything default with empty loader.conf. 

Sorry for this, next time will do one thing at a time, not a few in
parallel.

Thank you.