kern/106316: Dummynet with multipass ipfw drops packets when reloading FW

[Cichas]

>Number:         106316
>Category:       kern
>Synopsis:       Dummynet with multipass ipfw drops packets when reloading FW
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Dec 04 13:50:04 GMT 2006
>Closed-Date:
>Last-Modified:
>Originator:     Cichas
>Release:        5.5-Release, 6.1-Release, 6.2-RC1
>Organization:
>Environment:
>Description:
IPFW2 with dummynet pipes 
net.inet.ip.fw.one_pass: 0
no skipto rule

After 'ipfw -f flush' you will get lots of:
ipfw: ouch!, skip past end of rules, denying packet

It's bug because at least kernel rule 65535 should be used for packets going out of pipes after rule flush.

>How-To-Repeat:
PC1 -- ethernet -- PC2
Run iperf test between them

On PC1 od PC2 run:
/sbin/sysctl net.inet.ip.fw.one_pass 0
ipfw pipe 1 config bw 256Kbit/s queue 512KBytes
ipfw pipe 2 config bw 256Kbit/s queue 512KBytes
ipfw add 10 pipe 1 ip from any to any out
ipfw add 10 allow ip from any to any out
ipfw add 20 pipe 2 ip from any to any in
ipfw add 20 allow ip from any to any in
/bin/sleep 20
ipfw -f flush
>Fix:
Only workaround:
/sbin/sysctl net.inet.ip.fw.one_pass=1
$fwcmd -f flush
$fwcmd add 65500 allow all from any to any

sleep XY to flush pipes, load new ruleset, one_pass=0 and delete rule 65500
>Release-Note:
>Audit-Trail:
>Unformatted: