PF frag entries limit reached on a server with hw.ncpu: 24

[Andreas Longwitz]

On one of my servers a saw some messages

   <kern.crit> dssinet kernel:
   [zone: pf frag entries] PF frag entries limit reached

The output of the command
   vmstat -z | grep "pf frag entries"
was
   pf frag entries:    40,  5000,    0,  5000,  18760,   0,   0

So there are 5000 free items, none is in use and the limit of 5000 is
reached. This situation is possible, when all the free 5000 items are
hold in the caches of e.g cpu 0,1..20 and now a thread running on cpu 21
wants to allocate an item of the zone pf_frent_z. The alloc fails
because of the limit and the zone allocater from time to time logs the
error message "limit reached".

After putting the line
          set limit frags 10000
in pf.conf the problem was gone, now I have 5700 free items and this
value did not change for some weeks.

For a server with 24 cpu the default PFFRAG_FRENT_HIWAT = 5000 was too
small. Maybe this default value should be adjusted.

Andreas