Re: pf cannot allocate memory after a time

On 2021-12-11 08:14, tech-lists wrote:
> Hi,
> 
> context: main-n251261-25d0ccbe101 on arm64.aarch64 (raspberry pi4b/8GB)
> 
> I'm trying to use pf with pf-badhosts
> (https://geoghegan.ca/pub/pf-badhost/latest/install/freebsd.txt) and am 
> seeing
> what *seems like* a pf problem which has been reported elsewhere in 
> different
> contexts ( e.g.
> https://forums.freebsd.org/threads/cannot-define-table-cannot-allocate-memory-since-upgrade-to-13-0.80822/
> )
> 
> from pfctl -sa:
> [...]
> LIMITS:
> states        hard limit   100000
> src-nodes     hard limit    10000
> frags         hard limit     5000
> table-entries hard limit 25400000 [*]
> [...]

On a system with 8Gb RAM and some quarter of a billion table entries 
(addresses) in
19 tables (default drop):

LIMITS:
states        hard limit   600000
src-nodes     hard limit    10000
frags         hard limit     5000
table-entries hard limit  1700000

For starters; reloading entire tables with high number of entries is a waste 
of resources.
As it will require 2 times the size of the table in resources during the 
load. It also
poses the potential to leave you completely vulnerable should you attempt to 
(re)load a
table with corrupted entries.
As such, and as a rule; I only ever add/subtract from tables as 
required/needed. Works
every time. In the case of a bad entry. pf(4) simply rejects the entr(y|ies) 
and carries
on w/o problems.
IMHO any program the requires you to reload entire tables is ill conceived. 
It should
only require you to manage the DIFFerences. I think that's the problem you're 
running into.

Either fix the program you're using or use/develop something else. :-)

HTH

-- Chris
> 
> [*] the pf-badhosts guide quotes 400000 for this value; I bumped it to 
> 25400000 in
> order to "give pf more memory"
> 
> The problem is that if pf tables either get reloaded or if the machine is 
> running
> for say over 24 hrs, pf throws errors. This works if the machine is rebooted 
> but
> pf isn't switched on:
> 
> [...]
> # doas -u _pfbadhost pf-badhost -O freebsd
>                                 Password:
> 
> pf-badhost 1512 - - Using experimental "aggy" aggregator...
> 
> 6105 addresses added.
> 6235 addresses deleted.
> 
> pf-badhost 1580 - -
> IPv4 addresses in table:  619200750
> [...]
> 
> running pfctl -e -f /etc/pf.conf loads and runs. A day or so later, I'll see 
> this
> in the logs, after pf-badhost runs its update:
> 
> [...]
> pf-badhost 15202 - - Using experimental "aggy" aggregator...
> 
> pfctl: Cannot allocate memory.
> 
> pf-badhost 15256 - - ERROR: '/etc/pf-badhost.txt' contains invalid data! 
> Reverting
> changes and bailing out...
> [...]
> 
> There's plenty of memory. I've tried running this with one term on top -P 
> open and
> there's always 1-2GB available (free) as well as 12GB of swap which is 
> unused.
> 
> If I try pfctl -Fa -f /etc/pf.conf and log back in and then run pf-badhost 
> manually:
> 
> [...]
> # doas -u _pfbadhost pf-badhost -O freebsd [...]
> 
> not only the pfbadhost table doesn't load but nothing loads:
> 
> [...]
> # pfctl -e -f /etc/pf.conf
> /etc/pf.conf:18: cannot define table pfbadhost: Cannot allocate memory
> /etc/pf.conf:23: cannot define table rfc6890: Cannot allocate memory
> /etc/pf.conf:26: cannot define table gooDNS6: Cannot allocate memory
> /etc/pf.conf:27: cannot define table friends: Cannot allocate memory
> pfctl: Syntax error in config file: pf rules not loaded
> [...]
> 
> The only solution is a reboot. How to fix? Do I need to increase 
> src-nodes/frags?
> 
> thanks,