Issues with PF and 7.1

[Max Laier]

On Friday 23 January 2009 18:21:32 Scott Ullrich wrote:
> On Thu, Jan 22, 2009 at 2:32 PM, Michael K. Smith - Adhost
>
> <mksmith at adhost.com> wrote:
> > Hello All:
> >
> > We are having memory issues with PF and 7.1p2 that we didn't experience
> > with 6.3.   Here's what happens.
> >
> > # pfctl -f /usr/local/etc/pf.conf
> > /usr/local/etc/pf.conf:135: cannot define table smtpd_reject_policyd:
> > Cannot allocate memory /usr/local/etc/pf.conf:139: cannot define table
> > smtpd_reject_spam: Cannot allocate memory pfctl: Syntax error in config
> > file: pf rules not loaded
> > # pfctl -t smtpd_reject_policyd -T flush
> > 94390 addresses deleted.
> > # pfctl -t smtpd_reject_spam -T flush
> > 62464 addresses deleted.
> > # pfctl -f /usr/local/etc/pf.conf
> >
> > So, after I flush the tables it loads.  Sometimes, however, we get a
> > global out of memory error " DIOCADDRULE: Cannot allocate memory "
> >
> > Here are my entries from pf.conf for various limits.  Everything else is
> > defaults.
> >
> > set limit tables 500
> > set limit table-entries 250000
> > set limit { states 1000000, src-nodes 300000, frags 100000 }
> > set optimization normal
> > set skip on lo0
> > set state-policy if-bound
> > set timeout interval 300
> > set timeout src.track 1200
> >
> > Finally, the box is using EM interfaces with VLAN's and has 4 Gig of
> > physical RAM.  There are two PF boxes in Active/Failover and the errors
> > show up on both, although they seem to show up more often on the Backup
> > device, which seems odd.
> >
> > Any help would be greatly appreciated.
>
> My first response would have been to set set limit table-entries but
> you already did that.
>
> Next thing I would check is a shot in the dark, but worth trying..
>
> What does sysctl vm.kmem_size_max show?   Try increasing that size a
> bit in loader.conf and see if that helps.

Seconded.  My guess is that the system flushes buffers when you first load the 
tables due to memory pressure, so when you load the tables a second time there 
is more space available.  This, however, suggest that you are pretty thin 
stretched regarding kvm and should really increase it.  I'd shoot for at least 
512M which I believe is the maximum in 7.1 with the stock kernel.  It seems 
that there is work in progress to increase that limit for amd64 in releng_7, 
however.  Increasing this is worthwhile in any case, as I have a hard time 
imagining what else you'd be doing with those 4G on the firewalls (unless you 
are running heavy webcaches on them, too).

-- 
/"\  Best regards,                      | mlaier at freebsd.org
\ /  Max Laier                          | ICQ #67774661
 X   http://pf4freebsd.love2party.net/  | mlaier at EFnet
/ \  ASCII Ribbon Campaign              | Against HTML Mail and News