ng_netflow/ipfw/bridge problems and Netflow best practices

[Ganbold]

Hi,

I'm newbie to Netflow and I'm trying to use ng_netflow because it is fast 
and uses less CPU.
I'm trying to collect Netflow traffic from FreeBSD 5.4 machine. Collector 
(flow-tools) runs on same machine.
This FreeBSD has 3 interfaces and it acts as bridging firewall using IPFW2.
It also uses dummynet.

host# uname -an
FreeBSD machine.mng.net 5.4-STABLE FreeBSD 5.4-STABLE #4: Fri Aug 12 
09:58:18 ULAST 2005     tsgan at machine.mng.net:/usr/obj/usr/src/sys/PRXY  i386

host# ifconfig
xl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
         media: Ethernet 100baseTX <full-duplex>
         status: active
xl1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
         media: Ethernet 100baseTX <full-duplex>
         status: active
vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         inet x.x.x.x netmask 0xffffffe0 broadcast x.x.x.x
         media: Ethernet autoselect (100baseTX <full-duplex>)
         status: active

I'm running ng_netflow module and ngctl with following parameters to catch 
both incoming and outgoing traffic:

ngctl mkpeer xl1: tee lower right
ngctl connect xl1: xl1:lower upper left
ngctl name xl1:lower xl1_tee
ngctl mkpeer xl1_tee: netflow left2right iface0
ngctl name xl1:lower.left2right netflow
ngctl connect xl1_tee: netflow: right2left iface1
ngctl msg netflow: setifindex { iface=0 index=2 }
ngctl msg netflow: setifindex { iface=1 index=1 }
ngctl mkpeer netflow: ksocket export inet/dgram/udp
ngctl msg netflow:export connect inet/127.0.0.1:8818

ngctl mkpeer xl0: tee lower right
ngctl connect xl0: xl0:lower upper left
ngctl name xl0:lower xl0_tee
ngctl mkpeer xl0_tee: netflow left2right iface2
ngctl name xl0:lower.left2right netflow0
ngctl msg netflow0: setifindex { iface=2 index=4 }
ngctl connect xl0_tee: netflow0: right2left iface3
ngctl msg netflow0: setifindex { iface=3 index=3 }
ngctl mkpeer netflow0: ksocket export inet/dgram/udp
ngctl msg netflow0:export connect inet/127.0.0.1:8818

However I have 2 issues.
1. Firewall dynamic rules count almost doubles when starts ng_netflow traffic.
2. Firewall behaves abnormally, customers complained that they couldn't 
connect to Internet.

Is this known issue? How can I fix those?

I rebooted firewall and I tried following:

ngctl mkpeer xl1: tee lower left
ngctl connect xl1: xl1:lower upper right
ngctl mkpeer xl1:lower one2many left2right many0
ngctl connect xl1:lower.left2right xl1:lower many1 right2left
ngctl name xl1:lower.right2left o2m
ngctl mkpeer o2m: netflow one iface0
ngctl name o2m:one netflow
ngctl mkpeer netflow: ksocket export inet/dgram/udp
ngctl msg netflow:export connect inet/127.0.0.1:8818

Same problems as before I had after that. I don't know yet how to solve 
these problems.

Can somebody in this list help me to solve above problems? Maybe somebody 
already had these issues and solved already.


Afterwards I tried softflowd and it is working fine except it adds 5% 
overhead to CPU. That is why I prefer ng_netfow instead of softflowd.
I'm using flow-tools and flowscan to collect traffic and make report using 
CUflow. Is there any better way to make nice graphs and reports? What other 
tools should I try? What is the best practice?

I appreciate if somebody can give me some hints and advices.
It would be great if someone can share configuration samples and best 
practices.

thanks in advance,

Ganbold