FreeBSD router - large scale

[Matthew Seaman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 27/05/2010 16:00:12, Kevin Wilcox wrote:
> Hello everyone.
> 
> We're in the very early stages of considering [Free|Open]BSD on
> commodity hardware to handle NAT *and* firewall duties for (what I
> consider to be) a sizable deployment. Overall bandwidth is low, only a
> gigabit connection, but we handle approximately fifteen thousand
> devices. DHCP and DNS would be passed through to other servers, this
> hardware would only be responsible for address translation and pf.
> 
> I've done this on a very, very small scale (small/home office, small
> business) but I'm curious how many other folks are doing it on this
> scale, the hardware they are running on and any "gotchas" they may
> have faced. Does pf on FreeBSD take advantage of multiple cores/SMP?
> Is it preferable, as with OpenBSD, to go for a very stout processor
> without much consideration to cores?  Would freebsd-net@ be a better
> place to ask this?
> 
> I'm getting ready to start digging in to memory and other resources
> needed based on available documentation but real-world usage is much
> preferred to my academic assessment.

I've used OpenBSD/pf + carp for several sites; also + relayd for a
reasonably high traffic website, plus various setups using IPSec
tunnels.  All very successfully.  On a reasonably fast modern processor,
PF can run pretty much at GB wirespeed for straight packet forwarding or
NAT.  Doing serious crypto slows things up somewhat.

The hardest job I've had an OpenBSD firewall do is actually as a
mid-level firewall between a DMZ full of web servers and a back-end
database layer.  The thing to watch out for is running out of states in
PF.  It's trivial to change that in the config, and given a machine with
1GB or so RAM dedicated to running PF, you can up the number of states
by a factor of a hundred or more without problem.  Also if you know all
your connections are from directly attached networks and very low
latency, you can be a lot more aggressive about dropping old states.

PF is basically single-threaded -- even on FreeBSD, multiple cores won't
help you a great deal.  (Unless you've got anything else running on the
firewall, when several cores is really useful, of course.)  On the other
hand, PF is not hugely CPU intensive.  Better to spend your money on the
best NICs you can afford. There are some useful enhancements in
OpenBSD-4.7/pf which haven't made it into FreeBSD yet -- FreeBSD pf is
basically equivalent to about OpenBSD-4.1 I think.
FreeBSD is compatible with more varieties of amd64/i386 based hardware,
and it does threading and multi-cpu very much better.

	Cheers,

	Matthew

- -- 
Dr Matthew J Seaman MA, D.Phil.                   7 Priory Courtyard
                                                  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey     Ramsgate
JID: matthew at infracaninophile.co.uk               Kent, CT11 9PW
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkv+mesACgkQ8Mjk52CukIyB4gCff56iOhw7jRwmH4jzhaRmZPiK
COwAoINJQZ8YRk3s4plAuoru4CIdQr/h
=xyZm
-----END PGP SIGNATURE-----