Re: "pfctl: Cannot allocate memory" issue with a large table

On 23.10.21 20:42, Marcel Bischoff wrote:

> I often run into the "pfctl: Cannot allocate memory" error upon
> replacing the table contents.
Hi,

I've encountered a similar issue after upgrading to FreeBSD 13.0. I have
even cherry-picked
https://github.com/freebsd/freebsd-src/commit/ea21980a3facfed4c2c6fd10d0f16276564fb540
which has not helped. I have a theory what is the problem here, but I
lack detailed knowledge to confirm it.

I have multiple Load Balancers running FreeBSD 11 or 13 and some of them
run with with only 6 or 8GiB of memory installed. Each one shows 1-3GiB
"wired" memory, <200MiB "active" memory and "inactive" slowly occupying
all available memory within weeks after boot.

Once there is only a few hundred MiB free memory, I can't reload the pf
ruleset anymore on FreeBSD 13. Most of memory allocations in pf happens
with M_NOWAIT flag. The aforementioned patch changes IOCTLs to request
memory with M_WAITOK, but this does not change memory allocated for
tables themselves.

My guess is that when memory is full of inactive pages, it becomes
impossible to allocate more UMA objects with M_NOWAIT, as it would
require first getting rid of those pages (swapping them out? freeing
them?). I'm unsure if this is due to changes in pf between 11 and 13, or
rather increased memory pressure from other parts of system. I've always
thought that it is beneficial to keep as much buffers / caches /
inactive stuff in memory for better performance, but apparently it makes
allocations which can't wait fail. Or at least that's my best guess,
which somebody more experienced in in-kernel memory management (as I
understand this would never be an issue in userspace!) should verify.

-- 
| pozdrawiam / greetings | Powered by macOS, Debian and FreeBSD |
|  Kajetan Staszkiewicz  |  www: http://vegeta.tuxpowered.net   |
`------------------------^--------------------------------------'