multiple uplinks from ISP

mc mc at netx.com.hk
Wed Mar 9 03:49:44 PST 2005 

In fact, the biggest problem with me is that I don't have any development 
machines for building a test network, in other words I cannot do experiments 
at anytime I want. I usually need to plan the experiment in details and do 
the experiment with some idle hot backup machine in the network, or I can 
also use things like VMware to setup a testing network, but getting VMware's 
network to work as expected is headache....

outgoing/incoming ratio: its the reverse actually :P. out/in is roughly 
equals to 10:1, usually 12Mbps incoming and 99Mbps outgoing. actually most 
the traffic is just generated by a single web server.

 > If it's like 1:10, maybe you won't need multipath routing. You will  use 
only one interface for sending packets, and you will get them back via two 
interfaces. Think about it. In this case - everything that you need is two 
equal-cost routes to your network on the ISP side. Remember the KISS idea 
:-)
imho this setup have several drawbacks...at least if the sending link fails, 
the packets would not automatically go to the other interface. also, 
firewalling could be made difficult if the packets are distributed like 
this.


----- Original Message ----- 
From: "Nickolay Kritsky" <Nickolay.Kritsky at astra-sw.com>
To: "mc" <mc at netx.com.hk>; <freebsd-net at freebsd.org>
Sent: Wednesday, March 09, 2005 19:24
Subject: RE: multiple uplinks from ISP


1. Yes I think that should be enough.
2. Um, that's a good question. I guess I don't know the answer.
you can ask quagga maintainer about the details of quagga multipath routing. 
Maybe it just changes the gateway, say 10 times in a sec? Maybe it patches 
kernel binary code, who knows? The best way to know would be to build some 
test environment. What you need is two machines with 3 interfaces each. One 
would emulate the ISP side, one will be your side. and test. Plug them in 
between of some IP link and see what happens with tcpdump and other tools.
And, as it suddenly came to my mind, there is another question: what is your 
outgoing/incoming traffic ratio? If it's like 1:10, maybe you won't need 
multipath routing. You will  use only one interface for sending packets, and 
you will get them back via two interfaces. Think about it. In this case - 
everything that you need is two equal-cost routes to your network on the ISP 
side. Remember the KISS idea :-)

Nick

-----Original Message-----
From: mc [mailto:mc at netx.com.hk]
Sent: Wednesday, March 09, 2005 1:58 PM
To: Nickolay Kritsky; freebsd-net at freebsd.org
Subject: Re: multiple uplinks from ISP


dst-ip is not supported on one side of the switch.
src-mac does not work too, due to the fact that this would lead to a biased
result, causing most of the traffic goes thru the first link.
dst-mac would not work as the machine is sending traffic to a single router.

> fxp0: 1.2.3.1/30
> fxp1: 1.2.3.5/30
> em0: 10.123.123.102/24
Does this imply I just need to ask my ISP for two /30 and two default
gateways and that's it? No other 'special' configuration or registration
procedures would be needed?

One more question, did you mean if I am to use quagga as the bgp daemon, I
don't need to apply some kernel patches for the eq cost multipath to work?
'coz if my memory serves, quagga or other routing daemons just
insert/delete/update the route entries in the kernel, they do not take part
in any packet routing decisions.


----- Original Message ----- 
From: "Nickolay Kritsky" <Nickolay.Kritsky at astra-sw.com>
To: "mc" <mc at netx.com.hk>; <freebsd-net at freebsd.org>
Sent: Wednesday, March 09, 2005 18:35
Subject: RE: multiple uplinks from ISP


Why can't you use dst-ip hashing? You are using /24 network for your client
machines, no? If FEC uses IP addresses for hashing that you are ok. If it
uses MAC addresses for hashing, you need to test something else.
Regarding your initial post here is my proposal:

fxp0: 1.2.3.1/30
fxp1: 1.2.3.5/30
em0: 10.123.123.102/24

Your ISP gives you 2 more /30 nets for your uplinks
You should have two default gateways on fxp0 and fxp1 (1.2.3.2 and 1.2.3.6
respectively)
ISP AS should have two routes to your network with the same weight.

Problem: FreeBSD natively does not support two different routes to the same
destination. AFAIK this is by design.
Solution: It can be solved using custom patch (I think I have seen such for
4.x systems) or using external routing daemon like quagga.

Nick

-----Original Message-----
From: mc [mailto:mc at netx.com.hk]
Sent: Wednesday, March 09, 2005 1:06 PM
To: freebsd-net at freebsd.org
Subject: Re: multiple uplinks from ISP


Hi,

I am using cisco 29xx and 3xxx switches. The problem with FEC is that I have
no way to use dst-ip hashing as the load balancing option on these two
switches, and that would cause biased utilization on a certain link only,
i.e. impossible to utilize 2*100=200Mbps.

and...if I were really to use FEC as the solution, I will need to get some
much expensive switches from cisco, which is quite unaffordable and imho
unnecessary in fact...


----- Original Message ----- 
From: "Nickolay Kritsky" <Nickolay.Kritsky at astra-sw.com>
To: "mc" <mc at netx.com.hk>; <freebsd-net at freebsd.org>
Sent: Wednesday, March 09, 2005 17:58
Subject: RE: multiple uplinks from ISP


hello

I do not think you should mess a lot with interdomain routing here. Such a
scenario (multiple uplinks from the same ISP) IMHO is better be solved on
the layer 2.
What you need is some technology that utilizes two Ethernet ports at once.
About a week or two ago on this list was discussed similar setup using Cisco
technology. Search for subject "ng_fec and Cisco 2931". I f your ISP is
using the switch/router that supports FEC, you could do this trick. Also
most 3com intelligent switches support aggregating links via multiple
100Mbit channels. If you have put 3com equipment on both sides of your
internet connection you'll can get what you want.

Hope that helps.
BTW the first and best thing to do is to ask such question to your ISP.

Nick


-----Original Message-----
From: mc [mailto:mc at netx.com.hk]
Sent: Wednesday, March 09, 2005 12:32 PM
To: freebsd-net at freebsd.org
Subject: Re: multiple uplinks from ISP


Hi,

The main problem is that I have no idea at all how should I setup
everything..and what do I need from my ISP......I just know it was possible,
but I can't recall the details inside, and a simple google did not return
anything helpful to me.

I agree with you that fbsd (or any other linux) is much better than cisco in
terms of stability. The cisco routers at my site are crashing like cron jobs
while the fbsd boxes usually have long uptimes. :)

off topic: I used to be a network admin some time ago, but no longer true
now....and unfortunately, in the past I had only very few chances to deal
with interdomain routing, mainly in lab.
I'm afraid I have forgotten everything by now :(


----- Original Message ----- 
From: "Goran Gajic" <ggajic at mail.sbb.co.yu>
To: <freebsd-net at www.freebsd.org>
Sent: Wednesday, March 09, 2005 6:01
Subject: Re: multiple uplinks from ISP


>
> Hi,
>
> I have used succesfuly FBSD 5.2.1 as BGP router and it is rock stable with
> quagga (check out www.quagga.net) - more stable then 30k $ Cisco 7206 :))
> Problem is  if you have AS and LIR and if you don't there are other
> solutions. Of course much depends is your uplink ISP willing to cooperate.
>
> Regards,
> gg.
>
>
>
>> Hi all,
>>
>> If I have the following on hand...
>> - 2 FastEthernet uplinks from ISP
>> - 1 GigabitEthernet port on my switch
>> - a subset of a /24 allocated by ISP
>> The gigabit ethernet link should be connecting to my internal network.
>
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>


_______________________________________________
freebsd-net at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"


_______________________________________________
freebsd-net at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"


_______________________________________________
freebsd-net at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"

More information about the freebsd-net mailing list