802.3ad?

[Brad]

On Tue, Mar 28, 2006 at 07:20:15PM -0500, Richard A Steenbergen wrote:
> On Tue, Mar 28, 2006 at 04:59:11PM -0500, Brad wrote:
> > On Tue, Mar 28, 2006 at 08:56:24PM +0000, Baldur Gislason wrote:
> > > Following an unrelated discussion about "interface grouping" in OpenBSD,
> > > I'd like to know if there are any known or planned implementations of LACP (802.3ad)
> > > interface teaming in FreeBSD?
> > > FreeBSD currently has etherchannel support, but to my knowledge that will only
> > > work for a link to a single switch and not provide the possibility of having layer
> > > 2 resiliance with paths to 2 switches in the same network.
> > > Any thoughts on this?
> > 
> > 802.3ad is the standards track replacement for EtherChannel, thus it is not what you
> > are looking for. What you are asking for is L2 failover. OpenBSD's trunk(4) is such
> > an implementation. I don't see anything in FreeBSD that would have the same
> > functionality.
> 
> Ah nothing says fun like watching systems people try to figure out 
> networking. As I read it, he's looking for what ye olde networking people 
> call "equal cost multi-path" (ECMP) layer 3 load balancing, which is of 
> course very different from layer 2 load balancing. :)

He was not asking for ECMP. He clearly asked for a failover mechanism
for two switches within the SAME Layer *2* domain.

> 802.3ad/etherchannel and L2 failover are actually the same thing, just 
> with the load balancing algorithm turned off. Simple classic etherchannel 
> is really nothing more than configuring both sides to know that these are 
> trunk ports, and turning off forwarding among trunk members (i.e. you 
> don't want a frame which came in one member to go out another member, 
> unless you like forwarding loops). Everything else is pretty much 
> automagic, you don't even need to have matching hashing algorithms on each 
> end, just do whatever you need to do to shove the packets down the member 
> ports as you see fit.

What does link aggregation have to do with trunk ports? Link aggregation
and trunk ports can be configured separately or in a combined fashion but
in this scenario there is no need or use for trunking. What you have
described is still not what trunk(4) does.

> 802.3ad is the standardization of Cisco's (tm) etherchannel, but the only 
> thing it really brings to the table is the negotiation protocol. Cisco 
> used a proprietary protocol called PagP, 802.3ad defines a standard 
> version of the same thing called LACP. Essentially all this does is signal 
> trunk membership information over the individual links, to prevent stupid 
> misconfigurations that blackhole X percent of your traffic. For example 
> say that you have 2 devices talking to each other with 4 trunk members, 
> and someone disconnects a port from one and plugs it into a third device 
> which has no knowledge of the trunks. The device that is still connected 
> to that forth port has no way to know it is blindly blackholing 1/4th of 
> the traffic going through this channel, without a signaling protocol. The 
> operation of the L2 channel itself is really no different w/LACP or PagP 
> or nothing, it just adds another mechanism besides "hey we don't have link 
> any more" to shut individual members off if they're up to no good (or if 
> for some reason you really don't want them utilized unless another link 
> goes offline).

802.3ad = EtherChannel

PAgP = LACP

PAgP is the negotiation protocol for EtherChannel. So LACP is not bringing
anything new to the table, except getting away from Cisco propreitary
standards, which is a really good thing.

> As I read the man page, trunk(4) is just a classic protocolless 
> implementation of simple etherchannel style trunking. The round robin 
> option is a Bad Idea (tm) though, unless you like reordering your packets 
> and pissing off tcp royally. I believe FreeBSD has a netgraph 
> implementation of this, though I've personally never used it.

In theory round robin is bad but in reality it is not as bad as is typically
made out, plus it also depends on the protocols being used over this link.
trunk(4) can also provide L2 failover with a primary uplink and one or more
backup uplinks. trunk(4) can also work in other scenarios like for example
failover between a wireless uplink and a Ethernet uplink, when the AP is
within the same L2 domain (bridging AP, as opposed to a router) as the
Ethernet uplink.

> At the present time I don't believe fbsd makes any effort to even consider 
> the possibility of supporting the installation of multiple nexthops for a 
> route, let alone how to do it correctly. Andre Oppermann is probably the 
> person to pester about that, I know he's been doing a lot of work recently 
> trying to bring fbsd's routing code into the 21st century. If you're bored 
> and looking for something to work on outside of the routing code, I think 
> both fbsd and obsd's L2 channeling implementations suck badly. :)

You can think it sucks all you want, but it makes me sleep sounder at night
knowing I can engineer my system with higher availability in mind with trunk(4)
in use than is currently possible with a FreeBSD/NetBSD/DragonFly system.