Multiple default routes on multihome host

[Nick Barnes]

At 2008-02-18 21:36:18+0000, Bill Moran writes:
> In response to Nick Barnes <Nick.Barnes at pobox.com>:
> 
> > I have a multi-home host: more than one IP address.  The addresses are
> > in separate subnets but run over the same ethernet segment (this is a
> > temporary situation while I switch an office network over from one
> > network provider to another).
> > 
> > I want packets from address A1 to be sent via gateway G1, but packets
> > from address A2 to be sent via gateway G2.
> > 
> > How do I do this?  Can I just have more than one default route?  I'm
> > remote from the machine in question, so I don't want to tinker with
> > the default route until I'm sure of the answer.
> 
> You can't have multiple default routes.  The fact that you want to is
> an indicator of incorrect network design, although it could be an
> artifact of the interim setup while you migrate things around.
> 
> I would suggest you ask yourself (and possibly the list) _why_ you think
> multiple default routes is necessary ... what is it that you're hoping
> to accomplish.  I'm guessing your looking for some sort of redundancy,
> in which case something like CARP or RIP is liable to be the correct
> solution.

I agree that this is probably my inexperience showing.

I have an office network which is switching leased line, from provider
P1 to provider P2.  I have a /25 from P1 and a (different) /24 from
P2.  I am doing the migration a few machines at a time: move a little,
test a little, etc.  I am dual-homing each host for a short period
while I am switching it over.  The dual-homing works just fine, over a
shared ethernet segment, except for the fact that I can only have one
default route.

This means that I am sending packets from an address given to me by P2
to P1's router (my existing default route).  As an experimental
matter, today, this does in fact work - these packets are getting to
their destinations, via P1 - but it looks a heck of a lot like
spoofing and I am half-expecting the wrath of P1 to descend on me.
Either that or for them to silently stop routing the packets.

I would rather send packets from the P2 subnet addresses to the P2
router, while the packets from the P1 subnet addresses keep going to
the P1 router.

Apparently I can do this with some IPFW cunning, but that seems like
overkill for what seems like it ought to be a common problem.

If I were in the office, I would gird my loins for a single hard
session on all the consoles, to do all the config changes at once,
abandoning the P1 addresses.  As it is, doing it remotely, I'm being a
little more tentative.

Nick B