Packet loss with traffic shaper and routing

[. at babolo.ru]

[ Charset ISO-8859-1 unsupported, converting... ]
> Very good. You're right!
> I inserted a rule to match all non-layer2 packets on the top of the 
> ruleset and interrupts dropped 10~20% immediately.
> Given that, I went to apply Julian's idea of grouping 'in' and 'out' 
> pipe rules to reduce the searching on the firewall and that gave me a 
> little bit more of performance.
> As interrupts were still hitting 60% mark, I did some more experiences:
> 
> Test 1: I changed all 'pipe' rules to 'allow' rules, so all packets were 
> allowed and no shaping was done. The pipes were still there, but there 
> were no rules pointing packets to them.
> Result: No difference. Interrupts are the same as before.
> Conclusion: It's not the shaping itself that slows the system.
> 
> Test 2: With the same ruleset of test 1, I just removed all pipes (ipfw 
> pipe flush).
As far as I understand traffic stops after pipe flush,
and this is reason for CPU goes down

> Result: Interrupts were only 20%!
> Conclusion: Lots of pipes bother the system. I didn't figure out why, 
> but it's not a coincidence. I tested several times to make sure.
> 
> Test 3: I applied Michael's idea of using 'mask src-ip' and 'mask 
> dst-ip' in the pipes to use them as a template for dynamic generated pipes.
> Result: Worked like a charm. Now I have only 18 pipes instead of 3200. 
> Interrupts are ~30%.
> Conclusion: The reduced number of pipes generated less system interrupts.
> 
> The only problem I noticed (so far) with this method is that if we have 
> more than 1 IP address to a single MAC address, each IP will be shaped 
> individually instead of share the same speed of the other(s) IP(s) with 
> the same MAC.
> 
> Anyway, I am very curious about the result of test 2. Why do the pipes 
> have influence on system performance if there is nothing passing through 
> them?