dummynet dropping too many packets

[rihad]

Robert Watson wrote:
> I would suggest making just the HZ -> 4000 change for now and see how it 
> goes.
> 
Been running for a few hours under these changed sysctls:
kern.clockrate: { hz = 4000, tick = 250, profhz = 4000, stathz = 129 }
net.inet.ip.dummynet.io_fast: 1
net.inet.ip.dummynet.hash_size: 512
net.isr.direct: 0
net.inet.ip.intr_queue_maxlen: 5000
kern.ipc.nmbclusters=111111

Current stats:
net.inet.ip.dummynet.search_steps: 190347891
net.inet.ip.dummynet.searches: 188979871
net.inet.ip.dummynet.io_pkt_drop: 0
net.inet.ip.intr_queue_drops: 0
Around 1800 entries in each of the two ipfw tables.
About 420 mbps of traffic passing through.

ipfw pipe's created with GRED queuing algorithm, if it matters:
ipfw pipe 64 config bw 64kbit/s mask dst-ip 0xffffffff gred 
0.002/1800/2000/0.1 queue 2000
etc.

0 global output packet drops per second.

top -SH:
last pid: 59120;  load averages:  0.87,  1.19,  1.27 
                             up 0+04:31:38  17:11:59
141 processes: 9 running, 115 sleeping, 17 waiting
CPU:  3.8% user,  0.0% nice,  3.0% system,  8.8% interrupt, 84.4% idle
Mem: 1302M Active, 1482M Inact, 350M Wired, 40K Cache, 214M Buf, 801M Free
Swap: 2048M Total, 2048M Free

   PID USERNAME   PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
    16 root       171 ki31     0K    16K CPU2   2 237:10 97.02% idle: cpu2
    14 root       171 ki31     0K    16K CPU4   4 225:33 96.48% idle: cpu4
    12 root       171 ki31     0K    16K CPU6   6 249:31 88.96% idle: cpu6
    15 root       171 ki31     0K    16K CPU3   3 224:18 88.92% idle: cpu3
    13 root       171 ki31     0K    16K CPU5   5 228:39 87.74% idle: cpu5
    11 root       171 ki31     0K    16K CPU7   7 235:02 87.35% idle: cpu7
    18 root       171 ki31     0K    16K RUN    0 198:11 81.98% idle: cpu0
    17 root       171 ki31     0K    16K CPU1   1 209:31 71.78% idle: cpu1
    21 root       -44    -     0K    16K WAIT   0  88:24 47.31% swi1: net
    29 root       -68    -     0K    16K WAIT   1  34:06 18.95% irq256: bce0
   470 root       -68    -     0K    16K -      7  21:24  6.01% dummynet
    19 root       -32    -     0K    16K WAIT   5  13:37  5.86% swi4: 
clock sio
    31 root       -68    -     0K    16K WAIT   2   7:45  4.44% irq257: bce1


I'll wait for the number of entries to reach 2000-2200 or so, and post 
back again.