PF in FreeBSD 5.3 versus 6.x

[Michael Conlen]

I am not using pfsync. I'm using a pair of foundry layer 7 switches  
to do firewall load balancing.

I've since set optimization to aggressive and have seen a reduction  
in packet loss.

One issue I've discovered is that mytraceroute 0.72 appears to be  
buggy with respect to statistics so I can't trust the results for  
standard deviation and mean response time. In particular the mean  
response time tends towards the minimum response time over time  
despite continuously higher numbers. Without an accurate mean there's  
no good way to get a idea of the distribution using mytraceroute, and  
I didn't use ping times before I made the switch. On the other hand  
my NTP server getting time from across the firewalls does show  
improvement in stability and jitter, and this tends to be the first  
application that shows network problems for me. The NTP server is  
tracking time to wtihin +300/-200 microseconds which is impossible  
with a unstable network.

With the change the state table is running around 20k entries.

Do you know if these issues are present in the betas of 7.0, which I  
understand is using pf 4.1?

--
Michael Conlen

On Oct 14, 2007, at 7:24 PM, Matthew Franz wrote:

> HI Michael,
>
> You don't say whether you are running pfsync because Bill Marquette
> (who I work with) and Max have been discussing a pretty nasty pfsync
> bug (on 6.2) on this list under high loads (probably starting where
> you are at in terms of pps throughput but going up to 70-90kpps) where
> the backup is unable to clear states and there is eventually a huge
> discrepancy between the master and the backup.
>
> If you are seeing this with a single box. Its on my list to try to
> reproduce this in the lab (and test some of the patches Max has
> developed) with smartbits but I still haven't had time. We are
> definitely seeing some PF losing state entries, but sort of assumed
> this was a pfsync issue (or an effect thereof) but if you are seeing
> this without pfsync, that would point to so more fundamental problems
> with PF under high load. I can also share so more specific stats
> offline if that would be helpful.
>
> - mdf
>
>
>
>
> On 10/9/07, Michael Conlen <m at obmail.net> wrote:
>> I've noticed at some point between 5.3 and 6.0 that PF seems to be
>> dropping more packets than with 5.3 and there is increased deviation
>> in latency. Using the same equipment handling about 25k PPS each way
>> I see about 0.3% packet loss with FreeBSD 6.2 and 6.0 with sub 0.1%
>> loss with FreeBSD 5.3.  Similarly the worst case response times for
>> ICMP packets is much less in 5.3 than in either version of 6.
>>
>> I'm using something pretty vanilla in terms of setup. No ALTQ support
>> or features, no redirects, just a lot of blocking and allowing. The
>> firewalls are using server class 3Com and Intel Gigabit (Fiber)
>> cards. The changes were noticed going forward and undone by going
>> back to FreeBSD 5.3 so I don't suspect physical problems at the  
>> moment.
>>
>> My pf.conf is essentially a block in all followed by a block in quick
>> against a table with 2000 entries, many of the /24 or /16 followed by
>> pass rules to the various host:ports we allow.
>>
>> If I login to the firewalls themselves and run mtr in each direction
>> I don't see any traffic loss. It's only when crossing the firewalls.
>>
>> Usage is about 25k packets per second and 100Mbit/sec 5 minute max
>> traffic. The switches are Foundry SI-800g.
>>
>> Also doing about 25k/sec searches with 400 inserts a second and 270
>> removals and 407 matches/sec. The state table seems to run about
>> 70,000 to 90,000
>>
>> Are there issues I should be aware of and should pf be able to handle
>> this kind of load?
>>
>> --
>> Michael Conlen
>>
>> _______________________________________________
>> freebsd-pf at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
>> To unsubscribe, send any mail to "freebsd-pf-unsubscribe at freebsd.org"
>>
>
>
> -- 
> Matthew Franz
> http://www.threatmind.net/