finding optimal ipfw strategy

[Victor Gamov]

On 27/08/2019 21:03, Andrey V. Elsukov wrote:
> On 26.08.2019 19:25, Victor Gamov wrote:
>> More general question about my current config.  I have about 200Mbit
>> input multicasts which bridged and filtered later (about 380 Mbit
>> bridged if trafshow does not lie me :-) )
>>
>> My FreeBSD box (12.0-STABLE r348449 GENERIC  amd64)  has one "Intel(R)
>> Xeon(R) CPU E31270 @ 3.40GHz"  and 4-ports  "Intel(R) PRO/1000
>> PCI-Express Network Driver".  HT disabled and traffic mainly income via
>> igb0 and out both via igb0 and igb2.  About 30 VLANs now active some at
>> igb0 and some at igb2.
>>
>>
>> And I have following `top` stat:
>> =====
>> CPU 0:  0.0% user,  0.0% nice, 80.5% system,  0.0% interrupt, 19.5% idle
>> CPU 1:  0.0% user,  0.0% nice, 34.1% system,  0.0% interrupt, 65.9% idle
>> CPU 2:  0.0% user,  0.0% nice, 17.1% system,  0.0% interrupt, 82.9% idle
>> CPU 3:  0.0% user,  0.0% nice, 46.3% system,  0.0% interrupt, 53.7% idle
>> =====
> 
> This doesn't look like heavy ipfw load.

Andrew,

I have 0.0% interrupt but 80.5% system load.  As this box hasn't any 
processes running (besides kernel + ntp + bsnmp) so I think this load 
produced by ipfw.

Also I think this load source may be packets processing by bridge: get 
one packet, bridge it (copy/malloc?) into many interfaces, drop packets 
on unnecessary ifaces (free?)

> E.g. this is top output from slightly loaded firewall (300Mbytes/s
> ~500kpps):

Yes, it's not a problem for 28 cores :-)

I have 4 cores only and about 700Mbit in/out via bridge


> last pid: 58184;  load averages:  9.07,  8.98,  8.83
>                                                                        up
> 72+07:45:55  21:01:36
> 821 processes: 36 running, 680 sleeping, 105 waiting
> CPU 0:   0.0% user,  0.0% nice,  0.0% system, 28.1% interrupt, 71.9% idle
> CPU 27:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
> 
> # pmcstat -S instructions -Tw1
> PMC: [INSTR_RETIRED_ANY] Samples: 443074 (100.0%) , 0 unresolved
> Key: q => exiting...
> %SAMP IMAGE      FUNCTION             CALLERS
>   39.2 kernel     sched_idletd         fork_exit
>   10.9 ipfw.ko    ipfw_chk             ipfw_check_packet
>    3.6 kernel     cpu_search_lowest    cpu_search_lowest
>    2.8 kernel     lock_delay           _mtx_lock_spin_cookie
>    2.5 kernel     _rm_rlock            in6_localip:1.3 pfil_run_hooks:0.6
>    2.2 kernel     rn_match             ta_lookup_radix:1.5
> fib6_lookup_nh_basic:0.6
> 
> As you can see, when ipfw produces high load, interrupt column is more
> than system.


-- 
С уважением,
Гамов Виктор