dummynet dropping too many packets

[rihad]

Peter Jeremy wrote:

> Since the problem only appears to manifest when table(0) exceeds 2000
> entries, have you considered splitting (at least temporarily) that
> table (and possibly table(2)) into two (eg table(0) and table(4))?
> This would help rule out an (unlikely) problem with table sizes.
It was quite easy to simulate without touching any code:
# ipfw table 0 list|wc -l
     3697
# move half the table into the new table(1)
# ipfw table 0 list|head -n1800|while read ip pipeid; do ipfw -q table 1 
add $ip $pipeid && ipfw -q table 0 delete $ip; done
# ipfw table 0 list|wc -l
     1899
# ipfw table 1 list|wc -l
     1800
# ipfw add 1061 pipe tablearg ip from any to table'(1)' out recv bce0 
xmit bce1

now the ipfw looks like this:
01060 pipe tablearg ip from any to table(0) out recv bce0 xmit bce1
01061 pipe tablearg ip from any to table(1) out recv bce0 xmit bce1

The drops are at 100-200 per second. It may have descreased by 100-200 
drops.  Strangely, deleting rule 1061 instantly increases the drop rate 
to 300-400 up to 700/s, although there's still only 1980 entries in 
table(0), which is somewhat inexplicable.

I'll increase the bce queue maxlen value one of these days and see.

> 
> 01060 pipe tablearg ip from any to table(0) out recv bce0 xmit bce1
> 01061 pipe tablearg ip from any to table(4) out recv bce0 xmit bce1
> 01070 allow ip from any to table(0) out recv bce0 xmit bce1
> 01071 allow ip from any to table(4) out recv bce0 xmit bce1
> 
> (And I agree that re-arranging rules to reduce the number of repeated
> tests should improve ipfw efficiency).
> 
> The symptoms keep making me think "lock contention" - but I'm not sure
> how to measure that cheaply (AFAIK, LOCK_PROFILING is comparatively
> expensive).
> 
> Finally, are you running i386 or amd64?
>