Lots of weird PF behavior on 7.2-STABLE

[Linda Messerschmidt]

On Tue, Dec 15, 2009 at 3:33 PM, Peter Maxwell <peter at allicient.co.uk> wrote:
> Add in an explicit "pass all" rule at the start and set the
> log keyword on it.  Make sure *none* of the web traffic is hitting
> this rule.

> If the box isn't too loaded, you may try using "log (all)" on the pass
> rules (so that ALL packets are logged, not just the initial SYN) -
> that way at least you'd find out which rules those data packets are
> hitting, which would likely pin down the problem quite a bit.. those
> missing packets must have went somewhere ;-)  If it were me, that
> would be my preferable option if it was available.

> Barring that, I'd suggest simplifying your setup as much as possible -
> is there too much traffic to remove the "route-to" and try it against
> a single webserver?

> I'd also suggest removing the "scrub" directive until you have it
> working properly.

These are all great suggestions, but I will probably have to wait
until the middle of the night to try them just in case of catastrophe.
:)  I will report back what I find.

> Is the "state-policy" floating or if-bound?

I don't know what that means. :(  I'll check the manual.  But it's
whatever the default is; I didn't leave anything but IP addresses out
of the config file.

> Does the problem affect other services in a similar manner, can you
> replicate the exact same problem with the web servers with sshd, for
> example?

In theory it probably affects DNS as well (the other service being
load balanced), but since that's a UDP protocol.

When it happens to my ssh sessions, it seems to happen to all of them at once.

> What's annoying me is that I'm fairly sure I've seen this problem
> before, but for the life of me I can't remember what caused it :-(

Oh no!

> pf uses the last rule in the ruleset that matches for a given packet,
> so for a first instance setup I'd suggest putting an explicit "pass
> all" as the first rule, then any pass rules that do load-balancing and
> the like, then a list of block rules.  It makes it a lot easier to
> read and debug.  Then once it's working as you want, you can move the
> block rules up to the top and add in the "quick" keyword for
> performance purposes.

We could probably do that while nothing bad is happening, as long as
we get it done before the next time we get DDOS'd.  At such times, we
really don't want hundreds of megs of garbage going through the load
balance rule, so "quick" becomes pretty important. :)  But in the
everyday case, we can squeak by the other way long enough to test.

Thanks!