Packet loss with traffic shaper and routing

Tue May 2 18:06:11 UTC 2006

tpeixoto at widesoft.com.br wrote:

>Hello.
>I think I should give some 'real world' examples.
>
>
>/etc/rc.firewall:
>
>[Ss][Hh][Aa][Pp][Ee][Rr])
> setup_loopback
>
> . /etc/rc.shaper
>
> ${fwcmd} add 65000 pass all from any to any
> ;;
>
>
>/etc/rc.shaper:
>
>${fwcmd} pipe 1 config bw 512Kbit/s
>${fwcmd} pipe 2 config bw 512Kbit/s
>${fwcmd} add pipe 1 all from any to any MAC any 00:11:22:33:44:55 in
>${fwcmd} add pipe 2 all from any to any MAC 00:11:22:33:44:55 any out
>${fwcmd} pipe 3 config bw 256Kbit/s
>${fwcmd} pipe 4 config bw 256Kbit/s
>${fwcmd} add pipe 3 all from any to any MAC any 66:77:88:99:aa:bb in
>${fwcmd} add pipe 4 all from any to any MAC 66:77:88:99:aa:bb any out
>${fwcmd} pipe 5 config bw 128Kbit/s
>${fwcmd} pipe 6 config bw 128Kbit/s
>${fwcmd} add pipe 5 all from any to any MAC any 00:01:02:03:04:05 in
>${fwcmd} add pipe 6 all from any to any MAC 00:01:02:03:04:05 any out
>${fwcmd} pipe 7 config bw 512Kbit/s
>${fwcmd} pipe 8 config bw 1024Kbit/s
>${fwcmd} add pipe 7 all from any to any MAC any 06:07:08:09:0a:0b in
>${fwcmd} add pipe 8 all from any to any MAC 06:07:08:09:0a:0b any out
>${fwcmd} pipe 9 config bw 64Kbit/s
>${fwcmd} pipe 10 config bw 64Kbit/s
>${fwcmd} add pipe 9 all from any to any MAC any ab:cd:ef:00:11:22 in
>${fwcmd} add pipe 10 all from any to any MAC ab:cd:ef:00:11:22 any out
>  
>
OK, so, put the MACs in numerical order:

00:01:02:03:04:05
00:11:22:33:44:55
06:07:08:09:0a:0b
66:77:88:99:aa:bb
ab:cd:ef:00:11:22

work out MASKS that divide them into a binary set.

e.g.
1 skipto 10 all from any to not MAC 00:00:00:00:00:00/8
2 skipto 5 all from any to not MAC 00:01:00:00:00:00/16
3 pipe 1 ip from any to any
5 pipe 2 ip from any to any

10 skipto 12 all from any to not MAC 06:00:00:00:00:00/8
11 pipe 3 all from any to any
12 skipto 14 all from any to not MAC 66:00:00:00:00:00/8
13 pipe 4 all from any to any
14 pipe 5 all from any to any

now, if you continue this on, you will run 16 rules to divide the 1600 
rules up to find the right pipe.

>
>This example is for 5 clients. We have 1600.
>As you can see, there are 2 rules and 2 pipes per host, not 1600.
>
>
>If we try rc.firewall like this...
>
>setup_loopback
>${fwcmd} add 65000 pass all from any to any
>
>... we are ok. Interrupts are low.
>
>So, following your line of thought, I tried a simple test...
>
>setup_loopback
>${fwcmd} skipto 65000 ip from any to any MAC any any
>. /etc/rc.shaper
>${fwcmd} add 65000 pass all from any to any
>
>This way, the packets will never pass through shaper rules, but interrupts
>still get very high.
>  
>

I don't see how that proves anything

>Basically, we need a solution to shape each MAC address with its specifics
>download e upload speeds.
>Given the tests, I don't see how skipto can help, but if you believe that
>tablearg (which I am not familiar with) might help, we can try it with
>7.x.
>  
>

Tablearg only works with IP addresses.

>Thanks.
>
>
>  
>
>>oops, forgot to fix my cut-n- pastes.. corrected triage below..
>>
>>
>>Julian Elischer wrote:
>>
>>    
>>
>>>Julian Elischer wrote:
>>>
>>>      
>>>
>>>>tpeixoto at widesoft.com.br wrote:
>>>>
>>>>        
>>>>
>>>>>>That would do it..
>>>>>>
>>>>>>In all versions of FreeBSD
>>>>>>you can use the skipto rule to make sure that only a few rules are
>>>>>>run for any
>>>>>>address. Use it to to a binary search for the right pipe.'
>>>>>>carefully using 'skipto' and 'table' can make it efficient to do
>>>>>>very complex
>>>>>>filters like that.
>>>>>>
>>>>>>            
>>>>>>
>>>>>Sorry, but I didn't realized how to use that as we have to shape
>>>>>each user individually, i.e., each MAC address on the LAN has its
>>>>>own download and upload speeds.
>>>>>
>>>>>Could you clarify how to improve the situation with the tools you
>>>>>mentioned?
>>>>>          
>>>>>
>>>>
>>>>
>>>>
>>>>Assuming you can not use "tablearg" yet (it will make this REALLY EASY)
>>>>then if you have 30 IPs you want to shape from 1.1.1.1 to 1.1.1.30
>>>>        
>>>>
>>>
>>>
>>>then, consider the following example using IP addresses.
>>>
>>>      
>>>
>>>>
>>>>        
>>>>
>>>ipfw add 1000 skipto 1110 ip from any to 1.1.1.16/28
>>>ipfw add 1010 skipto 1032 ip from any to 1.1.1.8/29
>>>ipfw add 1012 skipto 1021 ip from any to 1.1.1.4./30
>>>      
>>>
>>>ipfw add 1013 [anything] ip from any to 1.1.1.0
>>>      
>>>
>>>ipfw add 1014 [anything] ip from any to 1.1.1.1
>>>ipfw add 1015 [anything] ip from any to 1.1.1.2
>>>ipfw add 1016 [anything] ip from any to 1.1.1.3
>>>
>>>
>>>ipfw add 1021 anything] ip from any to 1.1.1.4
>>>ipfw add 1022 [anything] ip from any to 1.1.1.5
>>>ipfw add 1023 [anything] ip from any to 1.1.1.6
>>>ipfw add 1024 [anything] ip from any to 1.1.1.7
>>>
>>>
>>>ipfw add 1032 skipto 1051 ip from any to 1.1.1.12./30
>>>
>>>ipfw add 1040 [anything] ip from any to 1.1.1.8
>>>ipfw add 1041 [anything] ip from any to 1.1.1.9
>>>ipfw add 1042 [anything] ip from any to 1.1.1.10
>>>ipfw add 1043 [anything] ip from any to 1.1.1.11
>>>
>>>
>>>ipfw add 1051 [anything] ip from any to 1.1.1.12
>>>ipfw add 1052 [anything] ip from any to 1.1.1.13
>>>ipfw add 1053 [anything] ip from any to 1.1.1.14
>>>ipfw add 1054 [anything] ip from any to 1.1.1.15
>>>
>>>
>>>ipfw add 1110 skipto 1132 ip from any to 1.1.1.24/29
>>>ipfw add 1112 skipto 1121 ip from any to 1.1.1.20./30
>>>ipfw add 1113 [anything] ip from any to 1.1.1.16
>>>ipfw add 1114 [anything] ip from any to 1.1.1.17
>>>ipfw add 1115 [anything] ip from any to 1.1.1.18
>>>      
>>>
>>>ipfw add 1116 [anything] ip from any to 1.1.1.19
>>>      
>>>
>>>ipfw add 1121 anything] ip from any to 1.1.1.20
>>>ipfw add 1122 [anything] ip from any to 1.1.1.21
>>>ipfw add 1123 [anything] ip from any to 1.1.1.22
>>>ipfw add 1124 [anything] ip from any to 1.1.1.23
>>>
>>>
>>>ipfw add 1132 skipto 1151 ip from any to 1.1.1.28./30
>>>
>>>ipfw add 1140 [anything] ip from any to 1.1.1.24
>>>ipfw add 1141 [anything] ip from any to 1.1.1.25
>>>ipfw add 1142 [anything] ip from any to 1.1.1.26
>>>ipfw add 1143 [anything] ip from any to 1.1.1.27
>>>
>>>
>>>ipfw add 1151 [anything] ip from any to 1.1.1.28
>>>ipfw add 1152 [anything] ip from any to 1.1.1.29
>>>ipfw add 1153 [anything] ip from any to 1.1.1.30
>>>ipfw add 1154 [anything] ip from any to 1.1.1.31
>>>
>>>
>>>
>>>
>>>
>>>now this example shows a binary search in IP space, written (including
>>>bugs) by hand
>>>but if you are willing to write a suitable perl script, you can
>>>generate a binary search in MAC address space
>>>just as easily. just sort them into order and search..
>>>
>>>I'm not going to try it by had, but for 1600 hosts you should only
>>>need to go through
>>>15 rules per host on average, instead of 1600 rules per host.
>>>that should cut down your ipfw cpu usage by 1/100
>>>
>>>
>>>
>>>      
>>>
>>>>freebsd.org"
>>>>        
>>>>
>
>  
>