kern/93829: Pfsync state time problem with CARP + Arp.Balance
C.Dornig
c_dornig at gmx.de
Sat Feb 25 06:30:24 PST 2006
>Number: 93829
>Category: kern
>Synopsis: Pfsync state time problem with CARP + Arp.Balance
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Sat Feb 25 14:30:04 GMT 2006
>Closed-Date:
>Last-Modified:
>Originator: C.Dornig
>Release: 6.0 Release
>Organization:
none
>Environment:
FreeBSD fw-cluster-1 6.0-RELEASE FreeBSD 6.0-RELEASE #4: Thu Feb 23 15:01:55 CET 2006 root at t-fw-cluster01:/usr/obj/usr/src/sys/CD-UNIX i386
and
FreeBSD fw-cluster-2 6.0-RELEASE FreeBSD 6.0-RELEASE #4: Thu Feb 23 15:01:55 CET 2006 root at t-fw-cluster01:/usr/obj/usr/src/sys/CD-UNIX i386
>Description:
HI,
I have a problem with CARP + pf + pfsync in arp.balance mode.
I have config 2 Cluster Routing / netfilter machines with carp + arpbalance.
The pf rule a the same on both server.
if the servers run in none arp.balance mode the rules are all fine and working perfektli.
But, if i turn on arp.balance than i become follow problem.
I made a ping (icmp packet) from my client pc (Client-LAN) to the Server behind the PF Cluster in other LAN.
The first packet goes through the PFCluster1 and the back packet goes through 6luster2. But, the state information from the first packet to the server is not fast enough on the PFCluster2 machine and because the pf rules, the back packet will blocked. The next packet from client to server will passed also the back traffic.
With out arp.balance the rule are ok, and all traffic will passed and the states will write correct. Only routing without pf are all ok.
I have made all network diagnostics. I have made tcpdump on all interfaces and the carps are all OK. Also pfsync packets will receive and send from each machine. The two machine can send and receive packet each other.
I think there is a time probleme from the pfsync. I mean that pfsync send too slow the state change to the other.
>How-To-Repeat:
To reproduce you must setup two machines with follow config:
you need two nic interfaces.
IP Range LAN1:
1.1.0.0/16 on interface em1
IP Range LAN2:
10.1.127.101 on interface em0 - manage ip.
10.3.155.0/25 on interface vlan155 -> em0
PFsync:
15.1.1.0/24 pfsync on fxp0 with crossover cable to machine 2.
Carps:
1.1.10.50
and
10.3.155.254
Gateway on side 1:
1.1.10.50
Gateway on side 2:
10.3.155.254
ifconfig output from machine 1:
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
options=b<RXCSUM,TXCSUM,VLAN_MTU>
inet6 fe80::240:d0ff:fe43:d986%em0 prefixlen 64 scopeid 0x1
inet 10.1.127.101 netmask 0xffffff00 broadcast 10.1.127.255
ether 00:40:d0:43:d9:86
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=8<VLAN_MTU>
inet6 fe80::250:8bff:fe66:9274%fxp0 prefixlen 64 scopeid 0x2
inet 15.1.1.1 netmask 0xffffff00 broadcast 15.1.1.255
ether 00:50:8b:66:92:74
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
fxp1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
options=8<VLAN_MTU>
ether 00:50:8b:66:92:75
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
options=b<RXCSUM,TXCSUM,VLAN_MTU>
inet6 fe80::240:d0ff:fe43:d987%em1 prefixlen 64 scopeid 0x4
inet 1.1.10.101 netmask 0xffff0000 broadcast 1.1.255.255
ether 00:40:d0:43:d9:87
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
pfsync0: flags=41<UP,RUNNING> mtu 1348
pfsync: syncdev: fxp0 syncpeer: 15.1.1.2 maxupd: 128
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33208
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
inet 127.0.0.1 netmask 0xff000000
vlan155: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
inet 10.3.155.10 netmask 0xffffff00 broadcast 10.3.155.255
inet6 fe80::240:d0ff:fe43:d986%vlan155 prefixlen 64 scopeid 0x8
ether 00:40:d0:43:d9:86
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
vlan: 155 parent interface: em0
carp100: flags=41<UP,RUNNING> mtu 1500
inet 1.1.10.50 netmask 0xffffff00
carp: MASTER vhid 10 advbase 1 advskew 0
carp101: flags=41<UP,RUNNING> mtu 1500
inet 1.1.10.50 netmask 0xffffff00
carp: BACKUP vhid 11 advbase 1 advskew 100
carp1551: flags=41<UP,RUNNING> mtu 1500
inet 10.3.155.254 netmask 0xffffff00
carp: BACKUP vhid 155 advbase 1 advskew 100
carp1552: flags=41<UP,RUNNING> mtu 1500
inet 10.3.155.254 netmask 0xffffff00
carp: MASTER vhid 255 advbase 1 advskew 0
Ifconfig from machine 2:
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
options=b<RXCSUM,TXCSUM,VLAN_MTU>
inet6 fe80::240:d0ff:fe43:d986%em0 prefixlen 64 scopeid 0x1
inet 10.1.127.102 netmask 0xffffff00 broadcast 10.1.127.255
ether 00:40:d0:43:d9:86
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=8<VLAN_MTU>
inet6 fe80::250:8bff:fe66:9274%fxp0 prefixlen 64 scopeid 0x2
inet 15.1.1.2 netmask 0xffffff00 broadcast 15.1.1.255
ether 00:50:8b:66:92:74
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
fxp1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
options=8<VLAN_MTU>
ether 00:50:8b:66:92:75
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
options=b<RXCSUM,TXCSUM,VLAN_MTU>
inet6 fe80::240:d0ff:fe43:d987%em1 prefixlen 64 scopeid 0x4
inet 1.1.10.102 netmask 0xffff0000 broadcast 1.1.255.255
ether 00:40:d0:43:d9:87
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
pfsync0: flags=41<UP,RUNNING> mtu 1348
pfsync: syncdev: fxp0 syncpeer: 15.1.1.1 maxupd: 128
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33208
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
inet 127.0.0.1 netmask 0xff000000
vlan155: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
inet 10.3.155.11 netmask 0xffffff00 broadcast 10.3.155.255
inet6 fe80::240:d0ff:fe43:d986%vlan155 prefixlen 64 scopeid 0x8
ether 00:40:d0:43:d9:86
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
vlan: 155 parent interface: em0
carp100: flags=41<UP,RUNNING> mtu 1500
inet 1.1.10.50 netmask 0xffffff00
carp: BACKUP vhid 10 advbase 1 advskew 100
carp101: flags=41<UP,RUNNING> mtu 1500
inet 1.1.10.50 netmask 0xffffff00
carp: MASTER vhid 11 advbase 1 advskew 0
carp1551: flags=41<UP,RUNNING> mtu 1500
inet 10.3.155.254 netmask 0xffffff00
carp: MASTER vhid 155 advbase 1 advskew 0
carp1552: flags=41<UP,RUNNING> mtu 1500
inet 10.3.155.254 netmask 0xffffff00
carp: BACKUP vhid 255 advbase 1 advskew 100
Then you need a little pf.conf with is same on both machines:
table <MANAGE> { 10.1.127.101 , 10.1.127.102 }
block log-all all
pass quick on lo0 inet from 127.0.0.1 to 127.0.0.1 keep state
### Pfsync Rule
pass quick on { em1 } proto pfsync
### CARP Rule
pass quick proto carp keep state
pass out log-all on em1 inet from 10.3.155.0/24 to 1.1.0.0/16 keep state
pass in quick log-all on em1 inet proto tcp from 1.1.0.0/16 to <MANAGE> port 22 keep state
pass in quick log-all on vlan155 inet from 10.3.155.0/24 to any keep state
pass out quick log-all inet from any to any keep state
Then you need 2 Test machines in LAN1 with IP:
1.1.XXX.YYY/16 and Gateway 1.1.10.50
Test Machines 2:
10.3.155.XXX/24 Gateway 10.3.155.254 -> with untagged vlanport.
And now you can test a ping from Test Machine to Test Machine.
Machine 1 must have the arp address from gateway 1
Machine 2 muss have the arp address from gateway 2
Only if the Machines has different MAC lists about her gateway, you can reproduce my problem.
>Fix:
I think you must change the source code for more pfsync packets send and receive.
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list