IPFW/Dummynet situation

[Oleg Bulyzhin]

On Tue, Mar 14, 2006 at 05:29:02PM +0100, Andrew Seguin wrote:
> I have a problem nagging at me for a while now...
> 
> If I create a pipe with a dst-ip mask (I haven't tried with a src-ip 
> mask) and a bandwith limit, the limit isn't respected properly. I know 
> it's not in the firewall rules themselves, the traffic goes into the 
> pipe, just when I use ipfw pipe show, I see more traffic then should 
> have been allowed, which is starting to be problematic considering the 
> slow internet pipe here.
> 
> For example:
> 10 second averages show 5 users receiving closer to (and above) 300kbps. 
> I thought maybe it was just my mental conversion from bytes to kbit that 
> was wrong, but I calculated: 250kbit / 8 = 31.25KByte, so I shouldn't 
> see more then 31000bytes in a dump (310 000 bytes for a 10s dump, 3.1M 
> for a 100s dump, etc), yet it isn't so per the dumps below:
> 
> firewall# ipfw pipe 20 delete && ipfw pipe 20 config bw 250kbps mask 
> dst-ip 0x000000ff && sleep 10 && ipfw -s 4 pipe 20 show
> 
> 00020: 250.000 Kbit/s    0 ms   50 sl. 13 queues (64 buckets) droptail
>    mask: 0x00 0x00000000/0x0000 -> 0x000000ff/0x0000
> BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
> Pkt/Byte Drp
> 23 ip           0.0.0.0/0           0.0.0.215/0      541   393993 48 
> 38867 113
> 49 ip           0.0.0.0/0           0.0.0.177/0      568   392311 50 
> 50243  82
> 23 ip           0.0.0.0/0           0.0.0.151/0      419   359542 40 
> 34010  26
> 25 ip           0.0.0.0/0           0.0.0.217/0      396   356667 44 
> 41133  17
> 19 ip           0.0.0.0/0           0.0.0.147/0      589   338828 47 
> 24481  34
> 59 ip           0.0.0.0/0           0.0.0.251/0      299    97693  0    
> 0   0
> 14 ip           0.0.0.0/0           0.0.0.206/0       39     5878  0    
> 0   0
> 33 ip           0.0.0.0/0           0.0.0.225/0       34     5039  0    
> 0   0
> 
> 
> 100 second averages:
> A014# ipfw pipe 20 delete && ipfw pipe 20 config bw 250kbps mask dst-ip 
> 0x000000ff && sleep 100 && ipfw -s 4 pipe 20 show
> 00020: 250.000 Kbit/s    0 ms   50 sl. 28 queues (64 buckets) droptail
>    mask: 0x00 0x00000000/0x0000 -> 0x000000ff/0x0000
> BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
> Pkt/Byte Drp
> 23 ip           0.0.0.0/0           0.0.0.215/0     4820  3561827 47 
> 55472 1758
> 19 ip           0.0.0.0/0           0.0.0.147/0     3604  3171878  0    
> 0 126
> 25 ip           0.0.0.0/0           0.0.0.217/0     3876  2915746 45 
> 11570  71
> 49 ip           0.0.0.0/0           0.0.0.177/0     4845  2764112  5 
> 2482 138
> 23 ip           0.0.0.0/0           0.0.0.151/0     2828  2344594 41 
> 30362 212
> 59 ip           0.0.0.0/0           0.0.0.251/0     4670  1777891  0    
> 0  21
> ...
> 
> Even with a 1000 second average I still see/have one computer fairly 
> high above the limit:
> 
> A014# ipfw pipe 20 delete && ipfw pipe 20 config bw 250kbps mask dst-ip 
> 0x000000ff && sleep 1000 && ipfw -s 4 pipe 20 show
> 00020: 250.000 Kbit/s    0 ms   50 sl. 43 queues (64 buckets) droptail
>    mask: 0x00 0x00000000/0x0000 -> 0x000000ff/0x0000
> BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
> Pkt/Byte Drp
> 23 ip           0.0.0.0/0           0.0.0.215/0     48823 34909898 49 
> 39751 14002
> 25 ip           0.0.0.0/0           0.0.0.217/0     40294 30358282 23 
> 19611 1301
> ...
> 
> 
> So is this normal or is it caused by something I'm doing or maybe not?
> 
> Thank you for any info!
> Andrew
> 
> _______________________________________________
> freebsd-ipfw at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
> To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe at freebsd.org"

Tot_pkt/bytes fields are number of pkts/bytes _tried_ to get through the pipe.

Let's look on your 1st flow (1000s results):
ave pkt size = 34909898/48823 ~ 715 bytes
number of dropped packets is 14002, so 14002*715 ~ 10011430 bytes was dropped.
so average flow throughput was (34909898-10011430)/1000 ~ 24898byte/s ~ 194kpbs.
(if you do same calculation for your 1st flow in 10s result you will get
throughput ~ 244kbps).

P.S. having dst-mask 0x000000ff will cause problems if you'll try to shape more
than one /24 network using same pipe.

-- 
Oleg.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ipfw/attachments/20060315/820e0b0c/attachment.pgp