IPFW MAX RULES COUNT PERFORMANCE

[Ian Smith]

On Mon, 27 Apr 2009, Daniel Dias Gonçalves wrote:
 > What may be happening ? I'm with polling enabled on all interfaces, can you
 > influence ?
 > 
 > em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0x7000-0x703f mem
 > 0xdfa00000-0xdfa1ffff irq 16 at device 8.0 on pci4
 > em1: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0x7400-0x743f mem
 > 0xdfa20000-0xdfa3ffff irq 17 at device 8.1 on pci4
 > em2: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0x8000-0x803f mem
 > 0xdfb00000-0xdfb1ffff irq 16 at device 8.0 on pci5
 > em3: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0x8400-0x843f mem
 > 0xdfb20000-0xdfb3ffff irq 17 at device 8.1 on pci5
 > em4: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0x9000-0x903f mem
 > 0xdfc00000-0xdfc1ffff irq 16 at device 8.0 on pci7
 > em5: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0x9400-0x943f mem
 > 0xdfc20000-0xdfc3ffff irq 17 at device 8.1 on pci7
 > em6: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xa000-0xa03f mem
 > 0xdfd00000-0xdfd1ffff irq 16 at device 8.0 on pci8
 > em7: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xa400-0xa43f mem
 > 0xdfd20000-0xdfd3ffff irq 17 at device 8.1 on pci8
 > fxp0: <Intel 82551 Pro/100 Ethernet> port 0xb000-0xb03f mem
 > 0xdfe20000-0xdfe20fff,0xdfe00000-0xdfe1ffff irq 16 at device 4.0 on pci14
 > 
 > If I disable the polling, no network interface work, begins to display "em4
 > watchdog timeout".

Sorry, no ideas about polling, but this doesn't smell like just an IPFW 
issue.  I was pointing out that despite 20 times the CPU clock rate, 
probably at least 30 times CPU throughput and likely 10 times the tick 
rate, you appear to be suffering something like 30 to 900 times the 
increased latency to be expected by traversing 'too many' ipfw rules.

 > Ian Smith escreveu:
 > > On Fri, 24 Apr 2009, Daniel Dias Gonçalves wrote:
 > > 
 > >  > The latency in the interface em6 increased an average of 10ms to 200 ~
 > > 300ms
 > >  > Hardware:
 > >  > CPU: Intel(R) Xeon(TM) CPU 3.20GHz (3200.13-MHz 686-class CPU)
 > >  >  Logical CPUs per core: 2
 > >  > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
 > >  > cpu0: <ACPI CPU> on acpi0
 > >  > p4tcc0: <CPU Frequency Thermal Control> on cpu0
 > >  > cpu1: <ACPI CPU> on acpi0
 > >  > p4tcc1: <CPU Frequency Thermal Control> on cpu1
 > >  > cpu2: <ACPI CPU> on acpi0
 > >  > p4tcc2: <CPU Frequency Thermal Control> on cpu2
 > >  > cpu3: <ACPI CPU> on acpi0
 > >  > p4tcc3: <CPU Frequency Thermal Control> on cpu3
 > >  > SMP: AP CPU #1 Launched!
 > >  > SMP: AP CPU #3 Launched!
 > >  > SMP: AP CPU #2 Launched!
 > >  >  > real memory  = 9663676416 (9216 MB)
 > >  > avail memory = 8396738560 (8007 MB)
 > > 
 > > In that case, there really is something else wrong.  By my measurements,
 > > rummaging through most of >1000 rules on a old 166MHz Pentium to get to the
 > > icmp allow rules (ridiculous, I know) added about 2ms to local net pings
 > > via that box, ie 1ms each pass for about 900 rules, mostly counts.

cheers, Ian