Dummynet performance

[Rickard Dahlstrand]

Hi,

Never mind this. After recompiling the kernel and adding a bunch of new
parameters I got less than or 1 ms delay and up to 65mbit/s TCP bandwidth.
(Up to 93mbit/s with a larger window-size!!)

For anybody trying the same, try doing a new kernel with this configuration:

options         BRIDGE
options         IPFIREWALL
options         IPDIVERT
options         IPFIREWALL_VERBOSE
options         IPFW2
options         HZ=2000
options         DUMMYNET
options         DEVICE_POLLING

and add this to /boot/loader.conf

kern.ipc.nmbclusters="32768"
kern.ipc.nsfbufs="32768"

and to /etc/sysctl.conf

net.link.ether.bridge_cfg=fxp1:0,fxp2:0
net.link.ether.bridge_ipfw=1
net.link.ether.bridge=1
net.inet.tcp.delayed_ack=0

By the way, this is a 1Ghz Via machine and I get approx 20% cpu load getting
thru 93% TCP traffic. Memory is contant at 91M free of 128M and no swap.

FreeBSD rocks!!

Rickard.

----- Original Message ----- 
From: "Rickard Dahlstrand" <rd at tilde.se>
To: <freebsd-ipfw at freebsd.org>
Sent: Wednesday, January 14, 2004 11:03 PM
Subject: Dummynet performance


Hi,

I have a fresh FreeBSD 4.9 installation and have activated the ipfw and
dummynet kernel modules and configured two ports as a bridge. Everything
works fine except one thing. When I add a pipe that does any kind of bw
limiting, I can't get thru more than 5 mbit/s TCP on my bridged interface.
UDP and packets with larger tcp windows size work better. One issue I found
was the RTT. It's <1ms in all cases except when I add bw limiting, then it's
up to 15-20ms. Does the bw limiting add this much RTT? Here is the config
and iperf results.

ipfw -q flush
ipfw -q add allow ip from any to any via fxp0
ipfw -q add pipe 1 all from any to any via fxp1 in
ipfw -q add pipe 2 all from any to any via fxp2 in
ipfw -q pipe 1 config bw 20Mbit/s delay 0ms
ipfw -q pipe 2 config bw 20Mbit/s delay 0ms

>iperf -c 10.1.1.100
------------------------------------------------------------
Client connecting to 10.1.1.100, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1952] local 10.1.1.121 port 3540 connected with 10.1.1.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[1952]  0.0-10.0 sec  6.70 MBytes  5.60 Mbits/sec

>iperf -c 10.1.1.100 -w 100k
------------------------------------------------------------
Client connecting to 10.1.1.100, TCP port 5001
TCP window size:  100 KByte
------------------------------------------------------------
[1952] local 10.1.1.121 port 3518 connected with 10.1.1.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[1952]  0.0-10.0 sec  17.8 MBytes  14.9 Mbits/sec

If I do this:

ipfw -q flush
ipfw -q add allow ip from any to any via fxp0
ipfw -q add pipe 1 all from any to any via fxp1 in
ipfw -q add pipe 2 all from any to any via fxp2 in
ipfw -q pipe 1 config
ipfw -q pipe 2 config

I get this:

>iperf -c 10.1.1.100
------------------------------------------------------------
Client connecting to 10.1.1.100, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1952] local 10.1.1.121 port 3541 connected with 10.1.1.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[1952]  0.0-10.0 sec   108 MBytes  90.8 Mbits/sec

Another issue I have is the fact that the delay-function doesn't appear to
work. (Or am I doing something wrong)

ipfw -q flush
ipfw -q add allow ip from any to any via fxp0
ipfw -q add pipe 1 all from any to any via fxp1 in
ipfw -q add pipe 2 all from any to any via fxp2 in
ipfw -q pipe 1 config delay 10ms
ipfw -q pipe 2 config delay 10ms

Reply from 10.1.1.100: bytes=32 time=11ms TTL=128
Reply from 10.1.1.100: bytes=32 time=15ms TTL=128
Reply from 10.1.1.100: bytes=32 time=13ms TTL=128
Reply from 10.1.1.100: bytes=32 time=12ms TTL=128

Ping statistics for 10.1.1.100:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 11ms, Maximum = 15ms, Average = 12ms

If I normally have <1ms to this host and according to the config I should
have 20ms.

I would really like some help on these issues.

Regards, Rickard.