Bad routing performance on 500Mhz Geode LX with CURRENT, ipfw and mpd5 (was: ipfw, "ip|all" proto and PPPoE -- does PPPoE packets passed to ipfw?)

[YongHyeon PYUN]

On Thu, Aug 30, 2012 at 01:11:58PM +0400, Lev Serebryakov wrote:
> Hello, Ian.
> You wrote 30 августа 2012 г., 10:23:56:
> 
>  >>   Yep, I'll collapse my two-rule chains in one rule.
> IS> I guess if the issue persists, we may need to see more of your ruleset.
>   Not a problem at all, here it is:
>   http://lev.serebryakov.spb.ru/_sklad/firewall.ipfw
> 
> IS> Hmm, you shouldn't see ANY pppoe traffic on ng0, only on the interface
> IS> mpd5 uses to connect with your DSL modem/bridge.  Nor would you expect
>   Yep. I didn't see it. My question is, really: why vr1 (my physical
> interface, used to connect to my ISP) takes 50%+ of CPU when traffic
> is only 40mbit/s down and about 20mbit/s up (with many connections)? I
> was afraid, that all PPPoE traffic is inspected by ipfw and it causes
> additional CPU load.

It would be interesting to know whether there is any difference
before/after taskq change made in r235334.  I was told that taskq
conversion for vr(4) resulted in better performance but I think
taskq shall add more burden on slow hardware.
Pre-r235334 interrupt handler still has issues since it wouldn't
exit interrupt handler if there are any pending interrupts.
It shall consume most of its CPU cycles in the interrupt handler
under extreme network load.  If pre-r235334 shows better result,
you are probably able to implement interrupt mitigation by using
VT6102/VT6105's timer interrupt. I guess some frames would be lost
with the interrupt mitigation under high network load but other
part of kernel would have more chance to run important tasks.
Anyway, vr(4) controllers wouldn't be one of best choice for slow
machines due to DMA alignment limitation and driver assisted
padding requirement.

> 
>   Yes, it is only 500Mhz Geode LX, but it is only 40 mbit/s and
> 4.5Kpps in both directions, nothing like full 100Mbit or more, and
> I've learned "empirical" rule/heuristics about 1Gbit(!) per 1Ghz(!)
> for softrouters, So, theoretically, 40mbit should not be a problem at
> all for this hardware.
> 
>   And now I have not-working WiFi (this box is also AP) when wired
> traffic is high (wifi speed drops down to 100KB/s from 2.5-3MB/s
> without wired traffic), userland freezes under load (very bad with
> ULE, better with 4BSD), and inability to pass through 40Mbit in both
> directions simultaneously.
> 
> -- 
> // Black Lion AKA Lev Serebryakov <lev at FreeBSD.org>