Problems getting lagg to balance using lacp

[Shteryana Shopova]

On Wed, Jan 27, 2010 at 2:40 PM, Steven Hartland
<killing at multiplay.co.uk> wrote:
> I've just setup lagg on one of our servers and run some
> basic tests and although everything seems to be reporting
> correctly no balancing appears to be happening.
>
> The following is the setup:
>
> FreeBSD side:
> ifconfig_em0="up"
> ifconfig_em1="up"
> cloned_interfaces="lagg0"
> ifconfig_lagg0="laggproto lacp laggport em0 laggport em1"
> ipv4_addrs_lagg0="10.10.1.66/24"
>
> Cisco side:
> interface Port-channel2
> description SERVER:: test - lagg0
> switchport
> switchport access vlan 96
> switchport mode access
> no ip address
> end
>
> interface GigabitEthernet3/35
> description SERVER:: test - em0
> switchport
> switchport access vlan 96
> switchport mode access
> no ip address
> channel-protocol lacp
> channel-group 2 mode active
> end
>
> interface GigabitEthernet3/43
> description SERVER:: test - em1
> switchport
> switchport access vlan 96
> switchport mode access
> no ip address
> channel-protocol lacp
> channel-group 2 mode active
> end
>
> Config:-
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>       options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
>       ether 00:30:48:33:ec:44
>       media: Ethernet autoselect (1000baseT <full-duplex>)
>       status: active
> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>       options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
>       ether 00:30:48:33:ec:44
>       media: Ethernet autoselect (1000baseT <full-duplex>)
>       status: active
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>       options=3<RXCSUM,TXCSUM>
>       inet 127.0.0.1 netmask 0xff000000 lagg0:
> flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>       options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
>       ether 00:30:48:33:ec:44
>       inet 10.10.1.66 netmask 0xffffff00 broadcast 10.10.1.255
>       inet 10.10.1.83 netmask 0xffffffff broadcast 10.10.1.83
>       media: Ethernet autoselect
>       status: active
>       laggproto lacp
>       laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
>       laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
>
> show lacp neighbor                              Flags:  S - Device is
> requesting Slow LACPDUs       F - Device is requesting Fast LACPDUs
>       A - Device is in Active mode       P - Device is in Passive mode
> Channel group 2 neighbors
>
> Partner's information:
>
>         Partner Partner   LACP Partner  Partner   Partner  Partner
> Partner
> Port      Flags   State     Port Priority Admin Key Oper Key Port Number
> Port State
> Gi3/35    SA      bndl      32768         0x0       0x90     0x1
> 0x3D  Gi3/43    SA      bndl      32768         0x0       0x90     0x2
>   0x3D
>
> Stats:
>
>     Interface           Traffic               Peak                Total
>         lagg0  in      0.351 KB/s        117.349 MB/s            2.695 GB
>                out     0.345 KB/s        114.183 MB/s            8.103 GB
>
>           em1  in      0.233 KB/s        117.349 MB/s            2.695 GB
>                out     0.000 KB/s        132.535 MB/s            8.890 GB
>
>           em0  in      0.116 KB/s          2.701 KB/s          331.405 KB
>                out     0.345 KB/s          9.093 KB/s          631.883 KB
>
>
> Name    Mtu Network       Address              Ipkts Ierrs    Opkts Oerrs
>  Coll
> em0    1500 <Link#1>      00:30:48:33:ec:44     5381     0     5188     0
>   0
> em1    1500 <Link#2>      00:30:48:33:ec:44  5987608     0  4015584     0
>   0
> lo0   16384 <Link#3>                               0     0        0     0
>   0
> lo0   16384 your-net      localhost                0     -        0     -
>   -
> lagg0  1500 <Link#4>      00:30:48:33:ec:44  5992849     0  4016823     8
>   0
> lagg0  1500 10.10.1.0   test           5985503     -  6475013     -     -
> lagg0  1500 10.10.1.83/ jail1       19     -        0     -     -
>
>
> Anyone got any ideas, as far as I can see lacp is negotiating correctly just
> no balancing is happening :(
>
>   Regards
>   Steve
>

What types of traffic streams are you testing this with? if_lagg will
use the SRC/DST MACs and IP addresses for IP traffic to decide which
member port of the lagg to sent the traffic out to, balancing of
incoming traffic should be done by the Cisco on the same principle.
Try a different laggproto for the if_lagg interface e.g. roundrobin
with channel-group <number> mode on set on the switchports.

cheers,
Shteryana