Packet loss with traffic shaper and routing

[tpeixoto at widesoft.com.br]

Very good. You're right!
I inserted a rule to match all non-layer2 packets on the top of the 
ruleset and interrupts dropped 10~20% immediately.
Given that, I went to apply Julian's idea of grouping 'in' and 'out' 
pipe rules to reduce the searching on the firewall and that gave me a 
little bit more of performance.
As interrupts were still hitting 60% mark, I did some more experiences:

Test 1: I changed all 'pipe' rules to 'allow' rules, so all packets were 
allowed and no shaping was done. The pipes were still there, but there 
were no rules pointing packets to them.
Result: No difference. Interrupts are the same as before.
Conclusion: It's not the shaping itself that slows the system.

Test 2: With the same ruleset of test 1, I just removed all pipes (ipfw 
pipe flush).
Result: Interrupts were only 20%!
Conclusion: Lots of pipes bother the system. I didn't figure out why, 
but it's not a coincidence. I tested several times to make sure.

Test 3: I applied Michael's idea of using 'mask src-ip' and 'mask 
dst-ip' in the pipes to use them as a template for dynamic generated pipes.
Result: Worked like a charm. Now I have only 18 pipes instead of 3200. 
Interrupts are ~30%.
Conclusion: The reduced number of pipes generated less system interrupts.

The only problem I noticed (so far) with this method is that if we have 
more than 1 IP address to a single MAC address, each IP will be shaped 
individually instead of share the same speed of the other(s) IP(s) with 
the same MAC.

Anyway, I am very curious about the result of test 2. Why do the pipes 
have influence on system performance if there is nothing passing through 
them?

Thank you very much everyone.


"."@babolo.ru wrote:
[...]
> In your example each packet walk through the rule set 4 times
>  1 mac input - abount half a ruleset average
>  2 ip input - all ruleset, not succesfull
>  3 ip output - all ruleset, not succesfull
>  4 mac output - abount half a ruleset average
> 
> allow all ip level packets on the ruleset begin and
> down proc usage 3 times down.
> 
[...]