"pfctl: Cannot allocate memory" issue with a large table

From: Marcel Bischoff <marcel_at_herrbischoff.com>
Date: Sat, 23 Oct 2021 18:42:35 UTC
Hi all,

for some time now I'm using the excellent (in my opinion) pf-badhost 
script [https://geoghegan.ca/pfbadhost.html] to create default 
blocklists for some servers. When using IPv6 and/or geoblocking with it, 
I often run into the "pfctl: Cannot allocate memory" error upon 
replacing the table contents.

The list contains about 300k+ lines with IPs and CIDRs. It is properly 
aggregated so the net blocks are compacted as far as possible into CIDR 
notation. Only single IPs are listed without a "/32" or "/128" suffix.

/etc/pf.conf contains

> set limit table-entries 1000000

/boot/loader.conf contains

> net.pf.request_maxcount=1000000
> kern.maxdsiz="2147483648"

/etc/sysctl.conf contains

> net.pf.request_maxcount=1000000

"pfctl -s memory" shows the limit is active:

> states        hard limit   100000
> src-nodes     hard limit    10000
> frags         hard limit     5000
> table-entries hard limit  1000000

During my research I found out that replacing a pf table temporarily 
needs double the memory as both the old and new states are held before 
the old is discarded. This makes entirely sense to me. What I don't 
understand is why the error still occurs despite the proper limit being 
set.

Does anyone have an idea how I can resolve this? It is entirely possible 
this happens due to me not entirely understanding how memory allocation 
in pf works. However, I haven't found anything particularly applicable 
either in the Handbook or the "pf.conf" man page.

Best,
Marcel