ipfw performance and random musings.

[Luigi Rizzo]

On Mon, Jul 31, 2006 at 02:15:56PM +0200, Ian FREISLICH wrote:
> Hi
> 
> I was wondering if anyone here had any ideas for improving the
> performance (packet rate) of ipfw.
> 
> I have about 500 interfaces on my firewall and I need to match and
> filter packets on a per interface basis.
> 
> I've found that while the server can move in excess of 360kpps
> bewteen arbitrary interfaces using about 5% CPU, if I turn on the
> firewall, my average packet rate falls off to about 60kpps on a UP
> system and 90kpps on a SMP system.  The maximum rate I can forward
> packets with ipfw enabled is 120kpps and that is with 1 rule allowing

this is a very strange number as it (120kpps with ipfw enabled)
is the performance i got in 2002 with a 750 MHz machine.
This was on 4.X - haven't checked recently on 6.x, there might
be some issue that has been introduced.
If you have a chance to try a 4.11 kernel (even a picobsd one)
on the same hardware it would be good to see some numbers.

> Perhaps these are 2 easy wins:
> 
> 1.  Change the order of the case statements in ipfw_chk() to move
>     more frequently used items to the top.  The options seem to

that has no impact in a sane compiler - the switch() is compiled
as a jump table and i don't see how gcc could do differently
given the small range of opcodes (6-8 bits).

> 2.  Caching of ifp->if_index in the rule 'microinstructions' to
>     remove the need for a strncmp to match interface names.  Might
>     be tricky if interfaces are destroyed and recreated without
>     invalidating this cache.

this is also a minor optimization - a strcmp is not that bad,
i think it is far worse to have to scan a long list of names.

> Then, state is not interface aware.  I have used this effect to

yes i agree that state is a bit limited, we only use the 5-tuple
but maybe we would need more info - the problem is, what more ?
do we always want an interface name or other metadata ?

this is a design problem in the first place.

	cheers
	luigi