kern/96715: very low network throughput between networks on systems with more than one nic

[Frank Rosenbaum]

>Number:         96715
>Category:       kern
>Synopsis:       very low network throughput between networks on systems with more than one nic
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed May 03 13:20:15 GMT 2006
>Closed-Date:
>Last-Modified:
>Originator:     Frank Rosenbaum
>Release:        6.0-RELEASE-p6
>Organization:
>Environment:
FreeBSD housing56.bytecamp.net 6.0-RELEASE-p6 FreeBSD 6.0-RELEASE-p6 #1: Tue Apr  4 04:34:39 CEST 2006     root at housing56.bytecamp.net:/usr/obj/usr/src/sys/BC-TUNED  i386
>Description:
The problem is the very low network throughput between the networks on systems with more than one nic. Five NICs are present in the system, two of them "fxp" and the other three "em". Polling is on all NICs enabled. We always set the connection type for each NIC in the /etc/rc.conf like ifconfig_fxp0="inet ... netmask ... media 100baseTX mediaopt full-duplex polling".
On a 100baseTX <full-duplex> connection we got 500 kilobytes per second as maximum throughput and the throughput always decreased during the tests. It does not matter if we use only fxp or em NICs for this test.
>How-To-Repeat:
Set up a system with more than 1 NIC. Set the connectiontype for all NICs in the /etc/rc.conf on "media 100baseTX mediaopt full-duplex" or directly with ifconfig, this should have the same effect.
Setup different networks on NICs and enable routing.
Now test the throughput between the networks.

It could be that only "fxp" and "em" NICs are affected, i do not have other NICs to test.
But we reproduce this bug on a new machine running an new installed RELENG 6_1 with four "fxp" NICs.
>Fix:
Delete the media and mediaopt options in the /etc/rc.conf so it looks like
ifconfig_fxp0="inet ... netmask ... polling"
Now the NICs use the autoselect function and ifconfig now say "media: Ethernet autoselect (100baseTX <full-duplex>)"
After this we got the "normal" throughput with more then 10 megabytes per second.
>Release-Note:
>Audit-Trail:
>Unformatted: