out xmit (demux) pipe bw accounting

Wed Aug 19 09:11:27 UTC 2009

Chuck Swiger wrote:
> Hi--
> 
> On Aug 17, 2009, at 7:32 AM, Michael Spratt wrote:
>> Could not find answer to following question.
>>
>> Given :
>> # pipe 1 config bw 1Mbit/sec
>> # ipfw 1000 add pipe 1 ip from any to any out xmit em0
>>
>> Will the bandwidth limit include layer 2 Ethernet headers or will only 
>> the IP datagram itself be included in the accounting mechanism?
> 
> IPFW normally processes traffic at layer-3, but you can toggle 
> net.link.ether.ipfw sysctl to on, in which case the above rule would 
> also be run against layer-2 packets including the 802.3 ethernet 
> header.  See the diagram in "PACKET FLOW" of "man ipfw"....
> 
> Regards,

Dear Chuck thanks for your response. I am not sure though that my 
question is answered. Yes switching net.link.ether.ipfw->1 does cause 
processing against layer 2 as indicated in the man ipfw diagram and the 
command
# sysctl -d net.link.ether.ipfw Yields the description 
net.link.ether.ipfw: Pass ether pkts through firewall

Setting this sysctl variable to 0 does disengage the processing of 
Ethernet frames and corresponding IP payloads according to the rule set 
specified and visa-versa.

I do not see anything indicating this sysctl variable affects the way 
dummynet does bit counting as it enforces bw limits on a pipe or queue.

Will the ruleset I provided above impliment the 1Mbit/s limit at layer 2 
including the Ethernet frames in the 1Mbit/s budget or (VS.) Only layer 
3 IP datagrams (and their coresponding layer 4 payloads) are 
incrementing the counters responsible for counting against the 1Mbit/s limit

Simplification of question:
  Given:
   # pipe 1 config bw 1Mbit/sec
   # ipfw 1000 add pipe 1 ip from any to any out xmit em0
  Dichotomy:
   layer 2 is limit to 1Mbit/s out em0 (Vs.) layer 3 is limit to 1Mbit/s 
out em0

Only one of these can be correct and I do not see documentation anywhere 
indicating to me which one is indeed correct. I am not educated enough 
to look in the source code to determine this for myself.

I did not state before the dummynet is on a freebsd transparent bridge.
Here are my sysctl variables

net.link.ether.ipfw: 0
net.link.bridge.ipfw: 1
net.link.bridge.ipfw_arp: 0
# sysctl -a | grep dummynet
net.inet.ip.dummynet.debug: 0
net.inet.ip.dummynet.pipe_byte_limit: 7000000
net.inet.ip.dummynet.pipe_slot_limit: 100
net.inet.ip.dummynet.io_pkt_drop: 33250421
net.inet.ip.dummynet.io_pkt_fast: 0
net.inet.ip.dummynet.io_pkt: 3512830485
net.inet.ip.dummynet.io_fast: 0
net.inet.ip.dummynet.tick_lost: 0
net.inet.ip.dummynet.tick_diff: 1662798
net.inet.ip.dummynet.tick_adjustment: 1679911
net.inet.ip.dummynet.tick_delta_sum: 721
net.inet.ip.dummynet.tick_delta: 2
net.inet.ip.dummynet.red_max_pkt_size: 1500
net.inet.ip.dummynet.red_avg_pkt_size: 512
net.inet.ip.dummynet.red_lookup_depth: 256
net.inet.ip.dummynet.max_chain_len: 16
net.inet.ip.dummynet.expire: 1
net.inet.ip.dummynet.search_steps: 0
net.inet.ip.dummynet.searches: 0
net.inet.ip.dummynet.extract_heap: 0
net.inet.ip.dummynet.ready_heap: 16
net.inet.ip.dummynet.curr_time: 1804818806
net.inet.ip.dummynet.hash_size: 64
# sysctl -a | grep fw
net.inet.ip.fw.dyn_keepalive: 1
net.inet.ip.fw.dyn_short_lifetime: 5
net.inet.ip.fw.dyn_udp_lifetime: 10
net.inet.ip.fw.dyn_rst_lifetime: 1
net.inet.ip.fw.dyn_fin_lifetime: 1
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_ack_lifetime: 300
net.inet.ip.fw.static_count: 26
net.inet.ip.fw.dyn_max: 4096
net.inet.ip.fw.dyn_count: 0
net.inet.ip.fw.curr_dyn_buckets: 256
net.inet.ip.fw.dyn_buckets: 256
net.inet.ip.fw.default_rule: 65535
net.inet.ip.fw.verbose_limit: 0
net.inet.ip.fw.verbose: 0
net.inet.ip.fw.debug: 1
net.inet.ip.fw.one_pass: 1
net.inet.ip.fw.autoinc_step: 100
net.inet.ip.fw.enable: 1

rule set

00900    15945359     1264958184 allow ip from 10.18.0.6 to any out xmit 
bce0
00901    44378882     8041056830 pipe 1 ip from 10.18.0.0/24 to any out 
xmit bce0
00910      225165       18913860 pipe 1 ip from 10.18.2.1 to any out 
xmit bce0
00915           0              0 pipe 2 ip from 10.18.2.2 to any out 
xmit bce0
00920           0              0 pipe 3 ip from 10.18.2.3 to any out 
xmit bce0
01000      265959       40261978 pipe 1 ip from 10.21.192.8/29 to any 
out xmit bce0
01001    19627389     3444748305 pipe 1 ip from 10.20.0.0/20 to any out 
xmit bce0
01002   180922838    31277916333 pipe 1 ip from 10.21.32.0/20 to any out 
xmit bce0
01003   376697028    75171259985 pipe 1 ip from 10.20.64.0/20 to any out 
xmit bce0
01004    68629348    12603497407 pipe 1 ip from 10.20.192.0/20 to any 
out xmit bce0
01005   185005607    37413644435 pipe 1 ip from 10.20.176.0/20 to any 
out xmit bce0
01006    59179395    10051795154 pipe 1 ip from 10.20.144.0/20 to any 
out xmit bce0
01007   235863676    41845371053 pipe 1 ip from 10.20.128.0/20 to any 
out xmit bce0
01008    51518999    10847386956 pipe 1 ip from 10.20.96.0/20 to any out 
xmit bce0
01009    79734709    13723937026 pipe 1 ip from 10.20.112.0/20 to any 
out xmit bce0
01010    30062283     4078276506 pipe 1 ip from 10.20.80.0/20 to any out 
xmit bce0
01011    36318511     6974450810 pipe 1 ip from 10.20.208.0/20 to any 
out xmit bce0
01012    19642496     3098922560 pipe 1 ip from 10.20.160.0/20 to any 
out xmit bce0
01013    18209570     2964820411 pipe 1 ip from 10.20.224.0/20 to any 
out xmit bce0
01014    13707931     3915527156 pipe 1 ip from 10.22.160.0/20 to any 
out xmit bce0
01015     9893051     1175739042 pipe 1 ip from 10.21.112.0/20 to any 
out xmit bce0
01016    39876049     7905051976 pipe 1 ip from 10.22.144.0/20 to any 
out xmit bce0
01017           0              0 pipe 1 ip from 10.22.147.0/24 to any 
out xmit bce0
01018   128251743    24972920481 pipe 1 ip from 10.21.0.0/20 to any out 
xmit bce0
01019   102099411    27172050432 pipe 1 ip from 10.21.16.0/20 to any out 
xmit bce0
65535 53581341116 24295389614128 allow ip from any to any

and pipe spec

# ipfw pipe list
00001:  11.500 Mbit/s    0 ms  75 KB 1 queues (1 buckets) droptail
     mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
   0 tcp     10.21.44.163/47934   65.55.131.121/80    1735939204 
358391598184  1  191 10152696

You can see that the bw has been configured to 11.5Mbit/s which should 
be the base 10 expression for bit/s meaning exactly 11,500,000 bit/s

And here lies the question again does this limit include layer 2 
Ethernet frame headers in the bit/s counting mechanism

-Mike

-- 
--------------------------------------------------

Mike Spratt - Iraq (GMT+3)

Office: (214)242-1782

DSN: (318)847-2983

Cell: +964-770-398-4366

mike at magicislandtechnologies.com

---------------------------------------------------