if_bridge + polling get lower througphts

[Chih-Chang Hsieh]

Thanks a lot.

We reset following MIBs:

net.link.bridge.ipfw=0
net.link.bridge.pfil_member=0
net.link.bridge.pfil_bridge=0

For non-polling mode, the interrupt rate is down to 30%,

For polling mode, the throughputs are as well as non-polling one

and the interrupt rate is less than 10%.



Andrew Thompson wrote:
> On Wed, Mar 15, 2006 at 06:16:29PM +0800, Chih-Chang Hsieh wrote:
>   
>> We have a FreeBSD 6.1-PRELEASE box.
>>
>> It runs with 2 em NICs and uses if_bridge + IPFW + ipfilter + pf.
>>
>> This box usually gets a very high interrupt rate >90%.
>>
>> By using netstat -I em0 1, we see:
>>
>> input (em0) output
>> packets errs bytes packets errs bytes colls
>> 15164 0 9549410 16842 0 12132262 0
>> 15331 0 9220723 17466 0 12354486 0
>> 16486 0 10089165 17926 0 13256014 0
>> 14936 10 8363433 17090 0 13367271 0
>> 14712 0 8449705 16701 0 13443079 0
>>
>>
>> When we enable the polling on em0 and em1 by ifconfig, netstat -I em0 shows:
>>
>> input (em0) output
>> packets errs bytes packets errs bytes colls
>> 10910 0 6629176 11467 0 7681346 0
>> 10951 0 6916042 11397 0 7954205 0
>> 11281 0 7437813 10804 0 7390386 0
>> 10711 0 6308033 11605 0 8400405 0
>> 10807 0 6270716 10973 0 7700748 0
>>
>> the interrupt rate decreases (<30%) but we get lower throuphts.
>>
>> net.link.bridge.ipfw: 1
>> net.link.bridge.pfil_member: 1
>> net.link.bridge.pfil_bridge: 1
>> net.link.bridge.pfil_onlyip: 0
>>
>> How could we tune the box to reach higher throughputs
>>     
>
> In the above setup you are running four packet filters, ipfilter, pf and
> IPFW twice (ipfw=1 is an extra layer2 run). More specifically the packet filters
> are actually invoked _10_ times per packet, in_if -> bridge_if -> out_if
> per pfil_hook, plus the extra ipfw layer2.
>
> This explains your high cpu usage, you may want to reorganise the
> filtering to optimise it.
>
>
> Andrew
>
>