if_bge upload stalls repeatedly (Was: 8-STABLE outgoing scp stalling frequently)

[Jonathan Chen]

On Wed, Feb 03, 2010 at 05:25:03PM -0800, Pyun YongHyeon wrote:
> On Thu, Feb 04, 2010 at 11:52:55AM +1300, Jonathan Chen wrote:
> > On Tue, Feb 02, 2010 at 11:20:29PM +0200, Nikos Ntarmos wrote:
> > > On Wed, Feb 03, 2010 at 08:36:16AM +1300, Jonathan Chen wrote:
> > > > Hi,
> > > > 
> > > > I've noticed that on a recent 8-STABLE/amd64, scp(1) appears to be
> > > > stalling very frequently. This is the output from a "scp -v -v"
> > > > of a 300Mb file from a local to a remote within an internal network:
> > 
> > [...]
> > > > Does anyone know what's happening here? Any tips on how to track down
> > > > what the problem is? The network config appears to be fine - fetch(1) will
> > > > have downloads speeds of up to 300KB/s.
> > > 
> > > But how about upload speeds? It seems that's where scp is suffering as
> > > well.
> > 
> > This is the obvious test that I should have done; and you're hit the
> > nail on the head. bge(4) on 8-STABLE (csup'd 4-Feb-2010) has a very
> > bad upload speed.
> > 
> > I've just tried using ftp to transfer some files:
> > 
> >     Upload speed: starts at 63 KB/s, falls rapidly before stalling.
> >     Download speeds: starts at 9 MB/s, increasing slightly before completing.
> I'm not sure but recently added code to support TSO may cause the
> issue. Would you show me verbose boot output(only bge(4) related
> one)?

bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x00a002> mem 0xf1bf0000-0xf1bfffff irq 17 at device 0.0 on pci9
bge0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xf1bf0000
bge0: adjust device control 0x2000 -> 0x5000
bge0: attempting to allocate 1 MSI vectors (1 supported)
bge0: using IRQ 258 for MSI
bge0: CHIP ID 0x0000a002; ASIC REV 0x0a; CHIP REV 0xa0; PCI-E
bge0: Disabling fastboot
bge0: Disabling fastboot
miibus0: <MII bus> on bge0
bge0: bpf attached
bge0: Ethernet address: 00:1d:09:d2:d1:9e
bge0: [MPSAFE]
bge0: [FILTER]
bge0: Disabling fastboot
bge0: Disabling fastboot
bge0: link UP

>To rule out possible TSO issue, disable TSO and try it
> again(#ifconfig bge0 -tso). Does it make any difference?

Yup, it sure does! With a TSO disabled, my upload and download speeds
are pretty much symmetrical at a decent 10MB/s.

Thanks!
-- 
Jonathan Chen <jonc at chen.org.nz>
----------------------------------------------------------------------
                                       Do not take life too seriously.
                                   You will never get out of it alive.