pfsync & carp problems

[Douglas K. Rand]

>> Some more information after I discovered the -x loud option to
>> pfctl. When the master firewall goes down and the already established
>> TCP session hangs, I get these messages on the slave:

>> pf: BAD state: TCP 67.134.74.224:52173 67.134.74.224:52173 204.152.184.134:80 [lo=2943781408 high=2943846943 win=33304 modulator=0 wscale=1] [lo=3255565389 high=3255629101 win=65535 modulator=0 wscale=0] 4:4 A seq=3255634893 ack=2943781408 len=1448 ackskew=0 pkts=21109:24835 dir=in,rev
>> pf: State failure on: 1       |

> This means the web server is trying to send data to the client that is
> out of (what pf thinks is legal for) its window.

> How are you disconnecting the master? Does this occur when you physically
> disconnect the ethernet cable towards the server first?

I've had my test TCP session hang by using both reboot and shutdown -r and
also by dropping the master into the kernel debugger and then after a few
minutes "cont"inuing.

> Ryan, do we address this, or is it just a rare but expected case that this
> might occur? Or did I miss anything and this shouldn't occur for some reason?

It doesn't see to rare to me. My test firewalls are forwarding packets for a
single TCP session. (A fetch of a FreeSBIE ISO.) Given two hours I'm confident
I can cause the problem to occur. (Admiditly in those two hours I'm causing
a failover far more often that production firewalls should see in a year 
or two. But, and maybe I'm guessing wrong here, I would expect that if a
single TCP stream has problems, I'm very likely to see a problem with
multiple established sessions.)

Thanks for the response.

If you have suggestions on further testing that I should do, I'm game.
Far easier now than after they go production. (If they do with pfsync.)