should if_lagg balance outbound traffic on an lacp connection ?

Andrew Thompson thompsa at FreeBSD.org
Wed Jul 4 18:31:31 UTC 2007


On Wed, Jul 04, 2007 at 03:44:06PM +0100, Pete French wrote:
> Having recently discovered if_lagg in stable I have spent the last couple
> of days experimentsin with this and using it to aggregate connections between
> a pair of servers and a pair of Cisco switches. It all appears to be
> functioning, but looking at the stats I see this:
> 
> Name    Mtu Network       Address              Ipkts Ierrs    Opkts Oerrs  Coll
> bce0   1500 <Link#1>      00:19:bb:33:00:de  7545268     0  9303330     0     0
> bce1   1500 <Link#2>      00:19:bb:33:00:de  7099168     0      126     0     0
> lo0   16384 <Link#3>                           15377     0    15376     0     0
> lo0   16384 fe80:3::1     fe80:3::1                0     -        0     -     -
> lo0   16384 localhost     ::1                     60     -       60     -     -
> lo0   16384 your-net      localhost            14222     -    14221     -     -
> lagg0  1500 <Link#4>      00:19:bb:33:00:de  7545187     0  9303208    58     0
> lagg0  1500 10.17.16/20   turpentine         7539218     -  9305974     -     -
> 
> Note that whilst I am getting evenly split traffic comming in from the switch,
> outgoing traffic is all heading out on bce0. The ifconfig output looks
> like this:
> 
>         options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
>         ether 00:19:bb:33:00:de
>         media: Ethernet autoselect (1000baseTX <full-duplex>)
>         status: active
>         lagg: laggdev lagg0
> bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
>         ether 00:19:bb:33:00:de
>         media: Ethernet autoselect (1000baseTX <full-duplex>)
>         status: active
>         lagg: laggdev lagg0
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
>         inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
>         inet6 ::1 prefixlen 128 
>         inet 127.0.0.1 netmask 0xff000000 
> lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         inet 10.17.19.0 netmask 0xfffff000 broadcast 10.17.31.255
>         ether 00:19:bb:33:00:de
>         media: Ethernet autoselect
>         status: active
>         lagg: laggproto lacp
>                 laggport bce1 =18<COLLECTING,DISTRIBUTING>
>                 laggport bce0 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
> 

bce1 is a concern here as it is not in the ACTIVE state. On your switch
have a look at the lacp stats, here is an example from mine with a 4
port aggregation.

c2950#sh lacp neighbor 
Flags:  S - Device is requesting Slow LACPDUs 
        F - Device is requesting Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode     

Channel group 1 neighbors

Partner's information:

                  LACP port                        Oper    Port     Port
Port      Flags   Priority  Dev ID         Age     Key     Number   State
Fa0/11    SA      32768     0005.5d71.8db8   4s    0xE6    0x1      0x3D  
Fa0/12    SA      32768     0005.5d71.8db8   5s    0xE6    0x2      0x3D  
Fa0/13    SA      32768     0005.5d71.8db8   4s    0xE6    0x3      0x3D  
Fa0/14    SA      32768     0005.5d71.8db8   3s    0xE6    0x4      0x3D  

c2950#sh lacp internal 
Flags:  S - Device is requesting Slow LACPDUs 
        F - Device is requesting Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode     

Channel group 1
                            LACP port     Admin     Oper    Port     Port
Port      Flags   State     Priority      Key       Key     Number   State
Fa0/11    SA      bndl      32768         0x1       0x1     0xB      0x3D  
Fa0/12    SA      bndl      32768         0x1       0x1     0xC      0x3D  
Fa0/13    SA      bndl      32768         0x1       0x1     0xD      0x3D  
Fa0/14    SA      bndl      32768         0x1       0x1     0xE      0x3D  



As you can see all ports are in 'bndl' state which means they have been negoiated.

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 00:05:5d:71:8d:b8
        media: Ethernet autoselect
        status: active
        lagg: laggproto lacp
                laggport ste3 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
                laggport ste2 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
                laggport ste1 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
                laggport ste0 =1c<ACTIVE,COLLECTING,DISTRIBUTING>


And all ports here are ACTIVE. I have been meaning to add IOCTLS to
display more lacp stats to help debug this sort of thing.


Andrew


More information about the freebsd-stable mailing list