Dummynet performance

[Rickard Dahlstrand]

Hi,

I have a fresh FreeBSD 4.9 installation and have activated the ipfw and dummynet kernel modules and configured two ports as a bridge. Everything works fine except one thing. When I add a pipe that does any kind of bw limiting, I can't get thru more than 5 mbit/s TCP on my bridged interface. UDP and packets with larger tcp windows size work better. One issue I found was the RTT. It's <1ms in all cases except when I add bw limiting, then it's up to 15-20ms. Does the bw limiting add this much RTT? Here is the config and iperf results.

ipfw -q flush
ipfw -q add allow ip from any to any via fxp0
ipfw -q add pipe 1 all from any to any via fxp1 in
ipfw -q add pipe 2 all from any to any via fxp2 in
ipfw -q pipe 1 config bw 20Mbit/s delay 0ms
ipfw -q pipe 2 config bw 20Mbit/s delay 0ms

>iperf -c 10.1.1.100
------------------------------------------------------------
Client connecting to 10.1.1.100, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1952] local 10.1.1.121 port 3540 connected with 10.1.1.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[1952]  0.0-10.0 sec  6.70 MBytes  5.60 Mbits/sec

>iperf -c 10.1.1.100 -w 100k
------------------------------------------------------------
Client connecting to 10.1.1.100, TCP port 5001
TCP window size:  100 KByte
------------------------------------------------------------
[1952] local 10.1.1.121 port 3518 connected with 10.1.1.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[1952]  0.0-10.0 sec  17.8 MBytes  14.9 Mbits/sec

If I do this:

ipfw -q flush
ipfw -q add allow ip from any to any via fxp0
ipfw -q add pipe 1 all from any to any via fxp1 in
ipfw -q add pipe 2 all from any to any via fxp2 in
ipfw -q pipe 1 config             
ipfw -q pipe 2 config             

I get this:

>iperf -c 10.1.1.100
------------------------------------------------------------
Client connecting to 10.1.1.100, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1952] local 10.1.1.121 port 3541 connected with 10.1.1.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[1952]  0.0-10.0 sec   108 MBytes  90.8 Mbits/sec

Another issue I have is the fact that the delay-function doesn't appear to work. (Or am I doing something wrong)

ipfw -q flush
ipfw -q add allow ip from any to any via fxp0
ipfw -q add pipe 1 all from any to any via fxp1 in
ipfw -q add pipe 2 all from any to any via fxp2 in
ipfw -q pipe 1 config delay 10ms
ipfw -q pipe 2 config delay 10ms

Reply from 10.1.1.100: bytes=32 time=11ms TTL=128
Reply from 10.1.1.100: bytes=32 time=15ms TTL=128
Reply from 10.1.1.100: bytes=32 time=13ms TTL=128
Reply from 10.1.1.100: bytes=32 time=12ms TTL=128

Ping statistics for 10.1.1.100:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 11ms, Maximum = 15ms, Average = 12ms

If I normally have <1ms to this host and according to the config I should have 20ms.

I would really like some help on these issues. 

Regards, Rickard.