Re: "pfctl: Cannot allocate memory" issue with a large table
Date: Tue, 26 Oct 2021 17:13:26 UTC
On 2021-10-23 11:42, Marcel Bischoff wrote: > Hi all, > > for some time now I'm using the excellent (in my opinion) pf-badhost script > [https://geoghegan.ca/pfbadhost.html] to create default blocklists for some > servers. When using IPv6 and/or geoblocking with it, I often run into the > "pfctl: > Cannot allocate memory" error upon replacing the table contents. > > The list contains about 300k+ lines with IPs and CIDRs. It is properly > aggregated > so the net blocks are compacted as far as possible into CIDR notation. Only > single > IPs are listed without a "/32" or "/128" suffix. > > /etc/pf.conf contains > >> set limit table-entries 1000000 > > /boot/loader.conf contains > >> net.pf.request_maxcount=1000000 >> kern.maxdsiz="2147483648" > > /etc/sysctl.conf contains > >> net.pf.request_maxcount=1000000 > > "pfctl -s memory" shows the limit is active: > >> states hard limit 100000 >> src-nodes hard limit 10000 >> frags hard limit 5000 >> table-entries hard limit 1000000 > > During my research I found out that replacing a pf table temporarily needs > double > the memory as both the old and new states are held before the old is > discarded. > This makes entirely sense to me. What I don't understand is why the error > still > occurs despite the proper limit being set. > > Does anyone have an idea how I can resolve this? It is entirely possible > this > happens due to me not entirely understanding how memory allocation in pf > works. > However, I haven't found anything particularly applicable either in the > Handbook > or the "pf.conf" man page. Have you reached your STATE limit? OTOH you might try adding the IPs from the list individually. Something like: iplist=" w.x.y.z a.b.c.d ... g.h.i.j " for block in $iplist do pfctl -T add -t <your-table-name-here> $block done I'm managing about a half dozen tables with a combined number of a over quarter of a billion addresses, and don't have a problem. Even on a servers with as little as 8GB RAM. HTH --Chris > > Best, > Marcel