ipfw dyn_buckets relation to dyn_max

[Andrew Kinney]

Hello,

I'm creating a firewall that will service a high traffic network. It 
is to replace an existing stateful firewall that maxes out at 64000 
simultaneous sessions (state table entries). The old firewall is a 
hardware appliance and that limit cannot be modified, so we're 
replacing it.

With our old firewall, an occasional well crafted DOS or dDOS could 
fill the state table and cause a network outage. We're hoping to 
avoid that with our new firewall.

I'm using ipfw and stateless rules where ever possible. We will still 
have some stateful rules in certain policies, though. We're using 
FreeBSD 8.0 amd64.

Obviously, we'll craft our rules in such a way to limit the number of 
dynamic rules allowed per IP to prevent simple DOS attacks from 
filling the state table, but we still want more than 64000 state 
table entries available. We have what we believe to be more than 
adequate CPU and RAM resources available.

The first obvious setting was:
# sysctl -w net.inet.ip.fw.dyn_max=524288
We're guessing at that number and will tune it as necessary to reach 
a balance between performance and potential for state table 
exhaustion.

The next setting we're not so sure about.
# sysctl -w net.inet.ip.fw.dyn_buckets=65536
I read through the /usr/src/sys/netinet/ipfw/ip_fw2.c code for more 
information, read the ipfw man page, and searched everywhere I could 
think of, but didn't find information that would help.

The man page has a self-referential explanation of the dyn_buckets 
value that isn't much help.

What effect does dyn_buckets have on the maximum number of dynamic 
rules? Is there a relation between the maximum number of dynamic 
rules and dyn_buckets? If so, what is it?

The source code seemed to indicate that dyn_buckets is related to the 
amount of memory used to store a hash table with a single list of 
dynamic rules. I am quite possibly misinterpreting the comments in 
the code there. It could have just as easily been interpreted as a 
single list of dynamic rules per bucket. I would have logically 
thought that it might be 1 bucket per parent stateful rule and one 
list per bucket, but the code and comments didn't seem to support 
that.

Without being able to read the code better (I only have entry level 
C++ skills), the exact use of dyn_buckets is unclear to me.

We know that dyn_buckets does not have a one to one relation to 
dyn_max because we regularly see dyn_count at 70000+ with the default 
net.inet.ip.fw.dyn_*_lifetime timer settings at our current level of 
traffic.

My main concern is that if dyn_buckets represents some kind of limit 
on the number of dynamic rules, I'd like to know it in advance and 
plan for it rather than have random dropped packets or a kernel 
panic.

I really appreciate any input I can get on this topic. The FreeBSD 
forums suggested this list was a better spot for this question when I 
posted the question there. I'm happy to read any documentation that 
will shed some light on this, if someone can direct me to it.

Sincerely,
Andrew Kinney
President and
Chief Technology Officer
Advantagecom Networks, Inc.
http://www.advantagecom.net
phone: 509-522-3696 ext. 101