IPFW DUMMYNET: Several pipes after each other

[Ian Smith]

On Thu, 22 Jan 2009 08:10:09 +0100 (CET)
 Sebastian Mellmann <sebastian.mellmann at net.t-labs.tu-berlin.de> wrote:

 > I'm using FreeBSD 7.0 with IPFW DUMMYNET enabled.
 > 
 > I've got a problem with creating a ruleset which allows me to limit the
 > overall bandwidth of a link and afterwards pass the packets to another
 > pipe for processing.
 > 
 > So far I've got those rules:
 > 
 > in_if="em0"
 > out_if="em1"
 > management_if="em2"
 > in_ip="100.100.100.1"
 > out_ip="200.200.200.1"
 > management_ip="172.16.0.201"
 > client1_subnet="192.168.5.0/26"
 > client2_subnet="192.168.6.0/26"
 > server_subnet="192.168.7.0/24"
 > 
 > download_bandwidth="6144Kbit/s"
 > upload_bandwidth="1024Kbit/s"
 > delay="0"
 > queue_size="10"

10 slots ie packets is likely too small a queue size at these rates.  
You want to check the dropped packet stats from 'ipfw pipe show' re 
that; see the section in ipfw(8) about calculating sizes / delays.

On the other hand, depending on how many hosts you're running individual 
queues for (mask 0xffffffff), you may need to trade with memory used ..

 > cmd="ipfw"
 > 
 > $cmd add 10 allow all from any to any via lo0
 > 
 > $cmd pipe 100 config mask src-ip 0xffffffff bw $upload_bandwidth queue $queue_size delay $delay
 > $cmd pipe 200 config mask dst-ip 0xffffffff bw $download_bandwidth queue $queue_size
 > 
 > $cmd add pipe 100 all from $client1_subnet to $server_subnet in via $in_if
 > $cmd add pipe 200 all from $server_subnet to $client1_subnet out via $in_if
 > 
 > $cmd add pipe 100 all from $client2_subnet to $server_subnet in via $in_if
 > $cmd add pipe 200 all from $server_subnet to $client2_subnet out via $in_if

I suggest using 'in recv' and 'out xmit' rather than via for these, for 
the sake of clarity.  'in recv' and 'in via' come to the same thing, as 
only the receive interface is known on inbound packets, but 'out via' 
applies to packets that were *received* on the specified interface as 
well as those going out on that interface after routing, which can lead 
to surprising results sometimes, and being more specific never hurts ..

 > $cmd add 10000 allow all from any to any via $management_if
 > $cmd add 20000 allow all from any to any via $in_if
 > $cmd add 30000 allow all from any to any via $out_if
 > 
 > ---
 > 
 > What I want to add now, is the possibility to limit the bandwidth of the
 > whole link, e.g. 100Mbit/s.
 > 
 > I've tried to add a pipe:
 > 
 > $cmd pipe 50 config bw 100Mbit/s queue $queue_size
 > $cmd add pipe 50 all from any to any via $in_if
 > 
 > But when I have a look at the pipes with 'ipfw show' I can only see
 > packets go through the pipe 50 and nothing goes through the other pipes
 > (which makes sense actually since IPFW work that way?).

IPFW works that way if you (likely) have net.inet.ip.fw.one_pass=1 .. so 
that packets exiting from pipes aren't seen by the firewall again.  If 
you set one_pass=0, packets are reinjected into the firewall at the rule 
following the pipe (or queue) action, which is what you want to do here.

And you'll surely need a much larger queue for this pipe, at 100Mbit/s.

cheers, Ian