carp+pfsync+freevrrpd+jail

Fri Jul 7 18:50:11 UTC 2006

Hello Dmitriy and thanks for your replay.

Dmitriy Kirhlarov wrote:

>>What i have is that when i'm pinging carp0 (inet) or carp1(lan)
>>interface's ip address of my firewall - i'm receivind DUP responses.
>>    
>>
>
>One from carp and other from freevrrpd.
>  
>
While this tests i have freevrrpd down. First of all i want to get fresh 
firewall with no bugs to continue to test ifstated/freevrrpd.

>>And when host2 is ths slave and i'm starting to ping carp0 address -
>>no traffic appears on master host - that means that the local carp
>>interface responding to my packets..
>>    
>>
>
>Yep. Full standby mode (backup don't have shared IP) is not
>implemented now.
>  
>
Hm... any alternative to carp in this case? (my idea below)

>>That means that in case some service (provided by jail managed by
>>freevrrpd) will be accessed from outside - i cannot be sure what
>>host will answer the request.
>>    
>>
>
>I don't understand your idea. Do you want to start-stop jail, when
>master node is down-up?
>  
>
Here we are.
We have two servers with shared disk subsystem (geom and/or external 
SCSI raid)

inet ----- em0-vlan101-carp0------------------carp3-vlan100-em0------lan1
                                     carp1                   |
                                     carp2                   
|----carp4-vlan200-em0------lan2

......................many vlans

inet ----- em0-vlan101-carp0-----------------carp3-vlan100-em0------lan1
                                    carp1                   |
                                     carp2                  
|----carp4-vlan200-em0------lan2

......................many vlans

carp0 - used for routing of traffice that passing from inet to vlans and 
from vlans to inet
Routing is one single task, so, maybe i do not need dedicated carp. But 
it will make ballance tuning mutch easy, by just changing carp interface 
parameters to change places for master and slave than to reconfigure 
services.
carp1 - used for routing to services that should (in normal state) work 
on host1
carp2 - used for routing to services that should (in normal state) work 
on host2

We have services: SMPT (postfix), cashe (squid+squidguard), ldap....... 
everything in jails,  with unique single IP address connected to loX 
interface each.
When one host become down it's IPs (and services) moving to another by 
freevrrpd/ifstated.
Routing of each single service IP made to the corresponding firewall 
carp(carp1 or carp2) interface.

In other words incoming packet from any interface should go to 
carp-master. In case service IP is local it just dealing with jail. But 
if it is remote it meets routing table and packet being forwarded to 
another host (where corresponding carp is the master and corresponding 
jail run)

Here what i have now
carp0 - 192.168.0.1/24 (host1 master)
carp1 - 192.168.1.1/24 (host1 master)
carp2 - 192.168.2.1/24 (host2 master)
carp3 - 192.168.3.1/24 (host1 master)
carp4 - 192.168.4.1/24 (host1 master)

jail1: ip: 192.168.10.1/32 (SMTP normaly run on host1)
jail2: ip: 192.168.10.2/32 (SQUID normaly run on host2)

on host2: route add host 192.168.10.1 192.168.1.1
on host1: route add host 192.168.10.2 192.168.2.1

Corresponding jail have to be on the host where corresponding carp is 
master.

If only master carp answering packets then packet, coming  from internal 
vlan200 (where hosts have carp's address set as default router) comes to 
host:
if local jail is running it's being directed to jail itself.
if no local jail - it meets routing table to be forwarded to 
corresponding carp1(2) IP. there it can find local jail IP and being 
forwarded there.

But problems arised when bouth, master and slave answerring to incoming 
packets.
First of all it is not clear why slave answering packets at all.
N.B. This is the IDEA. But i would like to make it reality :)

>>I have done some tests. When i'm sshing to virtual IP - sometimes
>>i'm getting ssh prompt and can login, and sometimes it says that
>>host auth info is bad (yes, because second server answering me at
>>this time) and sometimes i'm loosing ssh connection while session is
>>active.
>>    
>>
>
>Use 'advbase' and 'advskew' params on both hosts for hard managing
>status of nodes.
>
>  
>
>>No ballance needed. I want to have some service run in main OS, some
>>services run in jail and i want to be sure which host will answer
>>the request when bouth hosts are up and running.
>>    
>>
>
>Be careful, if you want to use carp IP inside jail. Only master node can
>get data from external world.
>I tried to make this schema:
>           ldap1
>	  /    \
>       ldapn1   ldapn2
>       | | |    | | |
>      clients  clients
>
>Where ldap1 -- master ldap server.
>ldapn1, ldapn2 -- jail on different hosts on carp shared IP. They
>connect to ldap1 and get fresh data from master ldap server.
>
>I found, that only master node can start successfully. Jail on backup
>node can't start, because ldap server can't start. It can't start,
>because, it can't successfully connect to ldap1.
>ldapn2 sends SYN, ldap1 returns SYN+ACK, and switch delivers this
>packet to ldapn1 -- active master node.
>In order to have this schema working, both nodes must have personal IP
>and shared IP, but it's impossible within current jail implementation.
>
>WBR
>  
>
Best regards,
Anton