Re: pf cannot allocate memory after a time
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Sat, 11 Dec 2021 18:12:44 UTC
Hi, On Sat, Dec 11, 2021 at 06:04:17PM +0100, Marcel Bischoff wrote: > I've stumbled into this issue some time ago as well. The usual remedies > of raising states and table-entries did not help. > > I could resolve this with a combination of increasing the process memory > limit and lowering ZFS memory usage. I'm guessing that you do use ZFS, > since you have an 8 GB Pi, if not please disregard. You're right, I am using zfs > FreeBSD limits the memory available per process to 512 MB by default. > It appears that large PF tables cause issues with this default. > Raising that limit to 2 GB seems to have done the trick. To attempt this, > add the "kern.maxdsiz" tunable to /boot/loader.conf and reboot, like so: > >kern.maxdsiz="2147483648" Yes, came to this conclusion shortly before you posted this. I've written what I've done further in https://forums.freebsd.org/threads/cannot-define-table-cannot-allocate-memory-since-upgrade-to-13-0.80822/#post-546253 I found this thread https://forums.freebsd.org/threads/pf-states-limit-reached.67337/ informative too. But the idea of increasing it came from https://lists.freebsd.org/pipermail/freebsd-pf/2011-May/006139.html I note that on recent -current (main-n251261) on aarch64 kern.maxdsiz seems to be 1073741824 by default. But on amd64 on releng/12.2-p7 and on releng/13.0-p5 and stable/13 it defaults to 34359738368 ! On armv6 r366954 12.2 release it's 536870912. I don't know why it's so low on current (arm64.aarch64). It would explain why i've only seen this issue on this arch. > Sometimes PF still tripped up when replacing one large table with another > (via pf-badhosts). Usually this happened when the generated list to replace > grew to several MB in size. I've read somewhere that PF needs double the > memory to replace a table since for an instant it needs to hold both tables > in memory, which makes sense to me. You can verify that this is the case by > first dropping and then recreating the pf-badhost table with all entries. > That usually works, while replacing (what pf-badhosts does) usually does not. Yes, can confirm it's the replacing where the errors arose. > In those cases I found the ZFS ARC cache to be the culprit. By default this > grabs most of the available memory to cache file access and frees it on application > request. Again, PF appears to require the memory quicker than ZFS appears to be > able to free it. Usually the default ARC settings are fine but in this case you > may want to tune them for testing purposes, again in /boot/loader.conf and reboot: > >vfs.zfs.arc_max="2000M" >vfs.zfs.arc_min="500M" Thank you for this, I'll do this if required. For now, I've set kern.maxdsiz=4294967296 and am monitoring the pfbsdhost's mailbox for error reports. -- J.