ipfw rules consuming CPU

[Sami Halabi]

on my box with 130 rules 100Mbit the cpu don't go above 5%.
I daily manage 1.5-6GB.

Thanks in advance,
Sami

On Sat, Jun 9, 2012 at 11:21 PM, Michael Spratt <
mike at magicislandtechnologies.com> wrote:

> I have Linux & FreeBSD systems running ipfw with 80 rules with 70Mb/s
> symmetric, passing traffic for about 1000-1200 hosts.
>
>
> Alexander V. Chernikov wrote:
>
>> On 09.06.2012 01:56, Sami Halabi wrote:
>>
>>> Hi,
>>>
>>> I Manage a FreeBSD server as an edge router&  firewall.
>>> the setup has 10G interfaces (ixgbe-82599EB) and 1G
>>> interfaces(em-82571EB&
>>> bce-BCM5709) connected to 10G/1G switches.
>>>
>>> With the following setup i get higher cpu usage:
>>> bce1-upstream provider with little bandwidth, so i use pipes to limit
>>> users, and subnets
>>> ix0 - Internet Exchange
>>>
>>> some rules.
>>> .
>>> .
>>> .from 4000 starts pipes for specefic ips bandwidth allocations
>>> 04000    6210053001    5845967300616 pipe 1003 ip from 182.46.92.13 to
>>> any
>>> out xmit bce1
>>> 04100   41289897537    3064110648124 pipe 1004 ip from any to
>>> 182.46.92.13
>>> in recv bce1
>>>
>> You should use pipe tablearg for that. Traversing 4k rules effectively
>> kills all performance.
>>
>>  .
>>> .
>>> .
>>> .7000 is the wider pipeline for the whole block
>>> 07000    9127154724    4651308720315 pipe 1000 ip from  182.46.92.0/24to
>>> any out xmit bce1
>>> 07100    4837016828     458027989917 pipe 1002 ip from any to
>>> 182.46.92.0/24 in recv bce1
>>> last rule default to accept...
>>>
>>> specefic pipes (1003-...) have limits say between 1-10Mbps, and the wider
>>> pipe (1000 and 1002) has a global limit of 40MBps that should be reached
>>> by
>>> all other non-specefic ips, config like this:
>>> #Wide
>>> ipfw pipe 1000 config bw 40Mbit/s queue 200Kbytes
>>> ipfw pipe 1002 config bw 40Mbit/s queue 200Kbytes
>>> #specefic
>>> ipfw pipe 1003 config bw 9Mbit/s queue 200Kbytes
>>> ipfw pipe 1004 config bw 9Mbit/s queue 200Kbytes
>>> ipfw pipe 1005 config bw 3Mbit/s queue 200Kbytes
>>> ipfw pipe 1006 config bw 3Mbit/s queue 200Kbytes
>>> ipfw pipe 1007 config bw 5Mbit/s queue 200Kbytes
>>> ipfw pipe 1008 config bw 5Mbit/s queue 200Kbytes
>>> ipfw pipe 1009 config bw 10Mbit/s queue 200Kbytes
>>> ipfw pipe 1010 config bw 10Mbit/s queue 200Kbytes
>>>
>>>
>>> with this configuration when i have lots of traffic (3-6GB) going via ix0
>>> (not necessarly the ips described above, lets say to a server in my net
>>> ip
>>> 1832.46.93.4 and users behind the Internet Exchange) i see high cpu usage
>>> (70-90%).
>>>
>>> my first test was to: ipfw add 1 allow all from any to any, and cpu usage
>>> drops immediatly to 10-15%.
>>> but that not why i want (i wantto keep thelimits) so I add rule right
>>> before 4000 and the cpu usage drops down to 10-20%:
>>> 03020 1669463072808 1493341413029803 allow ip from any to any via ix0
>>>
>>>
>>> Any advice why this happens? or should it be there in the first place?
>>> I use FreeBSD 8.1-R-p10-amd64.
>>>
>>> Thanks in advance,
>>>
>>>
>>
>>
>


-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert
FreeBSD SysAdmin Expert