Bacula File/Storage Connection Woes using PF

[Jeremy Chadwick]

On Wed, Mar 26, 2008 at 09:09:30AM +0000, Greg Hennessy wrote:
> Jeremy Chadwick wrote:
>> There's been too many cases I've experienced where using "keep state"
>> blindly results in state-mismatch increasing at a very fast rate.  When
>> I implemented this mentality on our production servers, our users
>> started pointing out that scp's between machines would randomly get
>> severed mis-stream, same with ssh sessions where large TCP windows were
>> used (such as doing 'dmesg' over and over):
>>
>> http://lists.freebsd.org/pipermail/freebsd-pf/2008-January/004050.html
>
> Which (taking a rough guess) looking at your rule set in the above has very 
> little to do with 'keep state' and a lot to do with 'modulate state'. IIRC 
> there is a filed bug which displays all of the aforementioned symptoms when 
> modulate state meets selective acknowledgement (SACK). I'm sure Max has the 
> gory detail, it may even be fixed.

Which is interesting for two reasons:

1) I never got an explanation in aforementioned thread, thus I had no
   knowledge of this bug until now -- thank you for telling me!

2) The documentation for FreeBSD pf is minimal.  The best we've got is
   pf.conf(5), /etc/pf.conf (which tells you to read pf.conf(5)), and
   /usr/share/examples/pf/* which doesn't give you much in the ways
   of education either.
   This forces users to turn to the official OpenBSD pf documentation,
   which strongly advocates the use of "modulate state" with TCP
   packets: http://www.openbsd.org/faq/pf/filter.html#state

With these facts at hand, surely you can see how someone would end up in
this situation?  And I am not the only one: Doug Sampson's mail is
further proof of this.

Now that you have educated me in regards to said bug, I will have to go
back and try using "keep state" instead.  Again, thank you!

>> The "use keep state on everything!" attitude seems to stem from people
>> reading the OpenBSD pf.conf documentation, which states that as of
>> OpenBSD 4.1, "keep state" is implicit on every rule (meaning it's done
>> whether you say "keep state" or not).  FreeBSD's pf isn't like this.
>>   
> You miss out the most important bit of the new PF 4.1 state keeping 
> defaults, 'flags S/SA'.

I assume "pf 4.1" means "OpenBSD 4.1's pf", or is there some internal
pf versioning number?

This brings up another situation: there's no version number of pf in
FreeBSD that I can find.  The OpenBSD docs continually say "as of
OpenBSD x.y".  This confuses people, who when using pf under FreeBSD,
have no knowledge of what version of pf we're using.  What version is in
RELENG_6?  7?  CURRENT?  I didn't know until a few minutes ago --
because I went to cvsweb and had to look up the CVS commit messages
myself:

http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/contrib/pf/net/pf.c

Now that I know, I can make appropriate adjustments.  But requiring
users to look at CVS commit messages is a bit unrealistic, don't you
think?  Maybe I should submit a PR asking that the version of pf pulled
into FreeBSD be kept in the pf(4), pf.conf(5), and pfctl(8) manpages?
What do you suggest?

> Our cousins over the road in the OpenBSD neighbourhood have done this 
> precisely because of the issues caused in prior versions of PF by using 
> stateless rules and/or establishing TCP state on anything other than the 3 
> way handshake.

Yep, aware of that -- except that users have no idea as to whether the
implicit "keep state" on every rule applies to FreeBSD or not, or if
it's "safe" or not, because OpenBSD != FreeBSD.  They read the OpenBSD
docs and go "errr... so what version is FreeBSD using?"

>> It gets more confusing when you consider the fact that even though UDP
>> and ICMP are stateless protocols, pf can keep track of their state too,
>> though I don't know if FreeBSD pf supports that (OpenBSD pf does).
>>   
> This is not a flame, but if you really do not know that, you really should 
> not be publicly advocating a position on the basis of incomplete 
> information.

And "incomplete information" is induced by what I described in my above
paragraphs.  Please don't be so harsh under the circumstances -- I was
going off of all the available information I had at the time.

-- 
| Jeremy Chadwick                                    jdc at parodius.com |
| Parodius Networking                           http://www.parodius.com/ |
| UNIX Systems Administrator                      Mountain View, CA, USA |
| Making life hard for others since 1977.                  PGP: 4BD6C0CB |