ARP behavior in FreeBSD vs Linux

[Peter van Dijk]

On Mon, Sep 19, 2005 at 03:34:10AM +0100, Gary Palmer wrote:
> There is another side effect, which comes into view with certain 
> configurations behind load balancers.  Foundry has an option (I believe 
> called "DSR" for Direct Server Return) which just fiddles with the MAC 
> address of the destination.  Other companies load balancers will 
> probably have the same option, but I've no idea what they'll call it. 

Linux Virtual Server calls it 'DR' for Direct Routing. I like this
feature a lot as it means our loadbalancer is basically idle :)

> connection and life is happy.  The return path from the host to the 
> originator bypasses the load balancer, and effectively halves the 
> traffic that the LB is having to process and do table lookups on, etc.  
> This obviously greatly increases the available capacity of the LB.

All true; except in most cases the win is much more than 50%.. compare
HTTP request size (<1KB) to HTTP response size (often >50KB) :)

> With a Linux box answering ARP as described above, it is possible that 
> the upstream router (or routers) COULD learn that the load balanced IP 
> actually belongs on one of the servers rather than the load balancer.  
> If that happens, your load balanced farm will quickly degrade and you'll 
> be scratching your head for hours to try and figure out whats going on.  
> Or the LB and the Linux box will get into an ARP war and random TCP 
> connections will get RSTs from the Linux box.

In setting up such a configuration, making sure the backend hosts do
not respond to ARP is always important; I've seen people assign the
frontend IP to normal ethernet interfaces on FreeBSD boxes and wonder
why it didn't work..

On FreeBSD, we solve this issue by assigning the IPs to lo0. For
Linux, this approach works equally well and is what the Linux Virtual
Server documentation recommends.

So, unless you have a weird policy of assigning these IPs to -other-
Ethernet interfaces, there is no problem on FreeBSD nor Linux :)

Cheers, Peter
-- 
peter at dataloss.nl        | ~ tonight tonight, what is this potion
http://blog.dataloss.nl/ | ~ that makes a fool of me
UnderNet/#clue           |     Wayfinder, fr-025 soundtrack