IPFW Dummynet Bridge Limiting

Wed Jul 12 22:48:34 UTC 2006

Hello Adam,

Wednesday, July 12, 2006, 9:13:11 PM, you wrote:

> Hey all,

>  

>             I have searched and searched and searched and can't seem to
> come up with the answer to this little mystery I have going on here.
> Maybe I could get some help from this large group of people who are much
> smarter than I am. I have a FreeBSD machine running 6.1-RC that has
> three NICs, two of which are acting as a bridge. It's a pretty standard
> setup. What I am attempting to accomplish is bandwidth limiting using
> dummynet over this bridge. Here's the network layout:

>  

> INTERNET ---- Core Router ---- Bridge (limiter) ---- Border Router ----
> Customer Base

>  

>             The reason for the bridge between two routers is because we
> also have our server farm between those routers. The customer base
> consists of multiple routed networks and they all get public IPs. The
> problem I'm having is that the bridge is not limiting any of the
> customer IPs. I see packets flowing through the IPFW rules but they're
> not being passed to the pipes. I will show the configuration
> momentarily. The weird thing is, I am able to unplug the Border Router
> from this whole setup and plug a laptop in to the bridge and set it up
> so the laptop IP is limited. This setup works fine and I can limit the
> laptop the way I expect the rest of the network to be. Here's my
> configuration with the Border Router plugged in and the 216.19.50.37 IP
> being used in the "Customer Base":

>  

> ---Kernel Config---

> options         SMP                     # Symmetric MultiProcessor
> Kernel

> options         IPFIREWALL              # Firewall support

> options         IPFIREWALL_DEFAULT_TO_ACCEPT

> options         IPDIVERT

> options         DUMMYNET                # Traffic limiting

> options         BRIDGE

> options         HZ=1000                 # strongly recommended by
> dummynet(4)

> device          apic                    # I/O APIC

>  

> ---Sysctl---

> net.inet.ip.fw.enable=1

> net.inet.ip.fw.one_pass=1

> net.link.ether.bridge_cfg=em0,em1

> net.link.ether.bridge.enable=1

> net.link.ether.bridge_ipfw=1

> net.inet.ip.fw.dyn_buckets=256

> net.inet.ip.fw.curr_dyn_buckets=256

>  

> ---rc.conf---

> defaultrouter="[mydefaultrouter]"

> hostname="[myhostname]"

> ifconfig_bge0="[mymanagementinterface]"

> cloned_interfaces="bridge0"

> ifconfig_bridge0="addm em0 addm em1 up"

> ifconfig_em0="up"

> ifconfig_em1="up"

> sshd_enable="YES"

> firewall_enable="YES"

> firewall_script="/etc/rc.firewall.bwmg"    # this just runs ipfw with
> the rules supplied in custom_firewall below

> firewall_quiet="NO"

> firewall_logging="YES"

> firewall_flags=""

>  

> ---ifconfig----

> -snip-

> em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500

>         options=8<VLAN_MTU>

>         ether 00:04:23:cb:60:aa

>         media: Ethernet autoselect (100baseTX <full-duplex>)

>         status: active

> em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500

>         options=8<VLAN_MTU>

>         ether 00:04:23:cb:60:ab

>         media: Ethernet autoselect (100baseTX <full-duplex>)

>         status: active

> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384

>         inet 127.0.0.1 netmask 0xff000000 

> bridge0: flags=8043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

>         ether ac:de:48:ce:fe:5c

>         priority 32768 hellotime 2 fwddelay 15 maxage 20

>         member: em1 flags=3<LEARNING,DISCOVER>

>         member: em0 flags=3<LEARNING,DISCOVER>

>  

> ---custom_firewall---

> -q flush

> -q queue flush

> -q pipe flush

> add 1 allow all from any to any via lo0

> add 2 deny all from any to 127.0.0.0/8

> add 3 deny all from 127.0.0.0/8 to any

> add 4 skipto 65534 all from any to any via bge0

> add 65534 allow all from any to any

> add 100 pipe 100 config bw 100Kbit/s

> add 10 pipe 100 all from any to 216.19.50.37 recv em0

>  

> # ipfw show 10

> 00010      11430        925353 pipe 100 all from any to 216.19.50.37
> recv em0

>  

> # ipfw pipe show 100

> 00100: 100.000 Kbit/s    0 ms   50 sl. 1 queues (1 buckets) droptail

>     mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000

> BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes
> Pkt/Byte Drp

>   0 icmp 216.109.112.135/0        216.19.50.37/0     11434   925679  0
> 0   0

>  

>             I have tried many different configurations including
> changing net.inet.ip.fw.one_pass to 0, changing the ipfw rule to recv
> and xmit on BOTH devices of the bridge, changing the ipfw rule from all
> to tcp and ip, and changing the rule from "any to 216.19.50.37" to
> "216.19.50.37 to any" (recv and xmit on both interfaces). I've also
> tried the kernel without IPDIVERT and with if_bridge. As I stated
> before, the odd thing is that when I plug directly into it with an IP of
> 216.19.0.225 (can't use the other one here) and modify the rules to
> reflect the new IP, the limiting works just fine. I have a feeling this
> is where the problem is, but I can't quite think of any reason why this
> wouldn't work. Previously, I had a Linux machine running TC installed in
> place of this machine but I personally prefer FreeBSD and feel ipfw is
> easier to configure than tc. The Linux machine worked just fine.

>  

>             Could anyone possibly help with this little problem? I'm
> stuck. Also, if I forgot to include any information, I apologize. I'm a
> bit spacey when I write emails. Just let me know what I missed and I can
> explain further. Thanks.

>  

> Adam

> _______________________________________________
> freebsd-ipfw at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
> To unsubscribe, send any mail to
> "freebsd-ipfw-unsubscribe at freebsd.org"

I dont't use it bridge but some thinks that can help u:
 1. use corect syctl variables form: net.link.ether.bridge.ipfw
 instead net.link.ether.bridge_ipfw (probably an wrong typing)
 2. read the end from man page about bridge, and
 net.inet.ip.fw.one_pass variable.
 "Also remember that bridged packets are accepted after the first pass
     through the firewall irrespective of the setting of the sysctl variable
     net.inet.ip.fw.one_pass, and that some ipfw(8) actions such as divert do
     not apply to bridged packets.  It might be useful to have a rule of the
     form

           skipto 20000 ip from any to any bridged
 "

 3. Luigi Rizzo say in his
 documentation: "there is always one pass for bridged packets"

-- 
Best regards,
 vladone                            mailto:vladone at spaingsm.com