FreeBSD-8.2: Channel Bounding: LACP or Roundrobin? With Cisco Catalyst

[Rumen Telbizov]

Denny,

Since LACP uses hashing to determine which channel to send the packet to
the traffic between two nodes (ip:mac ip:mac) will always get bound to only
one of the two channels.
I am using HP procurve's here and they do seem to hash by ip too
although I can't see/tweak that as in catalyst. If you can add the IP
address
in the hashing function of the switch then you have a better chance of
achieving
what you're trying to by bringing an IP alias (might have to try a few until
you
find one that maps to the opposite channel) and then having two iperfs
between
different IP addresses. In the real live scenario it will be equivalent to
having
multiple iscsi targets between the two hosts on top of different IPs if this
works
for you.

As for the load balancing option - my HP 2910al's seemed to get CPU
overloaded
when I push a gigabit of traffic with this option. My guess is the crazy mac
address
change might be exhausting the cpu much faster. I'd be interested to see how
this
affects catalyst - let me know.

Good luck,
Rumen Telbizov


On Fri, Apr 8, 2011 at 3:29 AM, Denny Schierz <linuxmail at 4lin.net> wrote:

> hi,
>
> I want to bound two e1000 (1Gb/s) channels and use at the moment LCAP,
> but the max throughput is slower, than without channel bounding. I've
> got round about 70MB/s instead of > 150MB/s - 200MB/s.
>
> I used iperf with standard options:
>
> :~$ iperf -f M -c 1.2.3.4
> ------------------------------------------------------------
> Client connecting to 1.2.3.4, TCP port 5001
> TCP window size: 0.02 MByte (default)
> ------------------------------------------------------------
> [  3] local 1.2.3.5 port 58637 connected with 1.2.3.4 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec    705 MBytes  70.5 MBytes/sec
>
> If a second PC do the same, than my 70MB/s splittet into ~30MB/s and
> ~40MB/s
>
> config:
>
> root at iscsihead-m:~# ifconfig lagg0
> lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
>
>
> options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
>        ether 00:15:17:f1:5d:5f
>        inet6 fe80::215:17ff:fef1:5d5f%lagg0 prefixlen 64 scopeid 0x5
>        inet 1.2.3.4 netmask 0xffffffc0 broadcast 1.2.3.255
>        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
>        media: Ethernet autoselect
>        status: active
>        laggproto lacp
>        laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
>        laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
>
>
> Config from the Cisco:
>
>
> cisco#sh run int po3
> Building configuration...
>
> Current configuration : 119 bytes
> !
> interface Port-channel3
>  description iscsi-test
>  switchport
>  switchport access vlan 111
>  switchport mode access
> end
>
>
>
> #sh etherchannel summary
> Flags:  D - down        P - bundled in port-channel
>        I - stand-alone s - suspended
>        R - Layer3      S - Layer2
>        U - in use      f - failed to allocate aggregator
>
>        M - not in use, minimum links not met
>        u - unsuitable for bundling
>        w - waiting to be aggregated
>        d - default port
>
>
> Number of channel-groups in use: 3
> Number of aggregators:           3
>
> Group  Port-channel  Protocol    Ports
>
> ------+-------------+-----------+-----------------------------------------------
> 1      Po1(SU)          -        Gi1/1(P)    Gi1/2(P)
> 2      Po2(SU)          -        Gi6/17(P)   Gi6/18(P)   Gi6/19(P)
> 3      Po3(SU)         LACP      Gi5/41(P)   Gi5/44(P)
>
> cisco#sh run int gi5/41
> Building configuration...
>
> Current configuration : 182 bytes
> !
> interface GigabitEthernet5/41
>  description iscsi-head1
>  switchport access vlan 111
>  switchport mode access
>  no cdp enable
>  channel-group 3 mode active
>  spanning-tree portfast
> end
>
>
>
> #sh run int gi5/44
> Building configuration...
>
> Current configuration : 183 bytes
> !
> interface GigabitEthernet5/44
>  description GRAU1_iscsi2
>  switchport access vlan 111
>  switchport mode access
>  no cdp enable
>  channel-group 3 mode active
>  spanning-tree portfast
> end
>
>
> #sh etherchannel load-balance
> EtherChannel Load-Balancing Configuration:
>        src-dst-mac
>
> EtherChannel Load-Balancing Addresses Used Per-Protocol:
> Non-IP: Source XOR Destination MAC address
>  IPv4: Source XOR Destination MAC address
>  IPv6: Source XOR Destination MAC address
>
>
> What I saw with tcpdump: it seems, that only one device is used.
>
> Maybe, Cisco uses mac and FreeBSD IP ?
>
> Any suggestions?
>
>


-- 
Rumen Telbizov
http://telbizov.com