Default router changes unexpectedly when dummynet enabled

[Özkan KIRIK]

Hi,

When I enable dummynet rules, default router changes unexpectedly.
If the pipe & queue rules not exists, there is no problem.
I tried all FreeBSD 7.1, 7.2, 7.3, 8.1-STABLE-201008 releases ( both
i386 and amd64 ).
All this FreeBSD versions are affected.

I inspected logs if someone logged in or changed route (with route -n
monitor command).
When the default route changed, there isnt any messages at the "route
-n monitor" command output.
I think there can be a buffer overflow in dummynet code.

This problem repeats once per 10 minute.
I wrote a shell script which monitors the default router.
I saw that sometimes netstat -rn shows that default router is changed
as 10.0.16.251 or 10.6.10.240 etc.
which are client IP addresses but routing still routes to
right router 193.X.Y.Z .
After a while, routing really fails.

You can access tcpdump capture file from:
http://193.255.128.30/~ryland/flowdata_10_0_16_251
This file captured while the default router changes.
Tcpdump capture, belongs to the IP Address which shown in default
router (10.0.16.251)

the tcpdump command:

tcpdump -w /home/flowdata_10_0_16_251 -ni bce0.116 host 10.0.16.251

dummynet rules are:
30000     pipe 3 tcp from 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 to
any dst-port 8000,80,22,25,88,110,443,1720,1863,1521,3389,4489 via em0
// Upload
30000     pipe 3 udp from 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 to
any dst-port 53 via em0 // Upload
30000     pipe 4 tcp from 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 to
any via em0 // Upload
30000     pipe 4 udp from 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 to
any via em0 // Upload
.... LOTS OF NAT RULES HERE (in kernel nat)
60000     pipe 1 tcp from any
8000,80,22,25,88,110,443,1720,1863,1521,3389,4489 to
10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 via bce0* // Download
60000     pipe 1 udp from any 53 to
10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 via bce0* // Download
60000     pipe 2 tcp from any to
10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 via bce0* // Download
60000     pipe 2 udp from any to
10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 via bce0* // Download

/sbin/ipfw pipe 1 config bw 8192Kbit/s mask dst-ip 0xffffffff
/sbin/ipfw pipe 3 config bw 1024Kbit/s mask src-ip 0xffffffff
/sbin/ipfw pipe 2 config bw 4096Kbit/s mask dst-ip 0xffffffff
/sbin/ipfw pipe 4 config bw 1024Kbit/s mask src-ip 0xffffffff

sysctl vars:
net.inet.ip.dummynet.hash_size=256 ( also tried with 8192 )
net.inet.ip.dummynet.pipe_slot_limit=500
net.inet.ip.dummynet.io_fast=1


Some Information about network:
System has 3 NICS as WAN, LAN, DMZ.
There are VLANs on WAN and LAN interfaces
Throuput between 20Mbps and 100Mbps.


Any ideas?

Regards,
Ozkan KIRIK
Mersin University @ Turkey