DELL PowerEdge 860 and Broadcom Gigabit Ethernet poor performance.

[Sten Daniel Soersdal]

Jean-Claude MICHOT wrote:
> The server is a DELL PowerEdge 860 freshly installed with
> FreeBSD 7.0-BETA4 (GENERIC Kernel).
> 
> pciconf and part of boot information:
> 
> bge0 at pci0:4:0:0:        class=0x020000 card=0x01e61028 chip=0x165914e4 rev=0x11 hdr=0x00
>     vendor     = 'Broadcom Corporation'
>     device     = 'BCM5721 NetXtreme Gigabit Ethernet PCI Express'
>     class      = network
>     subclass   = ethernet
> bge1 at pci0:5:0:0:        class=0x020000 card=0x01e61028 chip=0x165914e4 rev=0x11 hdr=0x00
>     vendor     = 'Broadcom Corporation'
>     device     = 'BCM5721 NetXtreme Gigabit Ethernet PCI Express'
>     class      = network
>     subclass   = ethernet
> 
> bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x4101> mem 0xfe5f0000-0xfe5fffff irq 16 at device 0.0 on pci4
> miibus0: <MII bus> on bge0
> brgphy0: <BCM5750 10/100/1000baseTX PHY> PHY 1 on miibus0
> brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
> bge0: Ethernet address: 00:1c:23:e1:78:7e
> bge0: [ITHREAD]
> bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x4101> mem 0xfe3f0000-0xfe3fffff irq 17 at device 0.0 on pci5
> miibus1: <MII bus> on bge1
> brgphy1: <BCM5750 10/100/1000baseTX PHY> PHY 1 on miibus1
> brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
> bge1: Ethernet address: 00:1c:23:e1:78:7f
> bge1: [ITHREAD]
> 
> There's no problem with input throughput (upto 980 Mbits) but output 
> throughput never go upper to 540 Mbits :(
> 
> Just for test, i have add to this server an Intel Gigabit Ethernet board 
> (em) and there's no problem to output data up to around 980 Mbits with
> this addon board.
> 
> If i boot the server with Linux Ubuntu, there's no output throughput 
> problem with Broadcom, so it seem to be FreeBSD bge driver problem.
> 
> I'm not the only one to have poor output performance with bge on        
> DELL PowerEdge 860
> http://lists.freebsd.org/pipermail/freebsd-net/2007-June/014373.html
> 
> I have also try various patch or setup driver default values
> http://lists.freebsd.org/pipermail/freebsd-net/2007-November/015951.html
> http://lists.freebsd.org/pipermail/freebsd-net/2007-November/015956.html
> 
> But all theses attempts to get better ouput performance was unsuccessful :(
> 
> Any idea are welcome.
> 
> PS: If it's usefull to debug and try to fix the problem, i can provide 
>     root access to a DELL PE860 test server with bge.
> 

Have you tried setting the tcp send and receive windows?
The defaults are:

net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536

Also you might want to try to lower:

net.inet.tcp.delacktime: 100



-- 
Sten Daniel Soersdal