local network throughput issues
usleepless at gmail.com
usleepless at gmail.com
Sat Aug 16 20:57:14 UTC 2008
On 8/16/08, Rick C. Petty <rick-freebsd2008 at kiwi-computer.com> wrote:
> Hello. I've been having some serious network throughput issues recently.
> Prior to these issues, I was running 7.0-STABLE from a few months back. I
> recently build a world/kernel from csup(1) on 2008-Jul-25 and started
> noticing the issues. I'm currently on a RELENG_7 from 2008-Aug-02 and am
> building another one today.
>
> The server is running samba, dnsmasq, an NFS server, and is the gateway for
> a couple of freebsd machines. I first noticed the issue when I was unable
> to play mp3s served over samba. I was getting a throughput of about 3
> KBytes/sec over my gigabit switched network. After a few weeks of trying
> different versions of samba and watching tcpdump/trafshow, I decided to try
> another test:
>
> % ssh gateway cat /dev/zero | dd of=/dev/null
> load: 0.00 cmd: dd 68286 [runnable] 0.00u 0.00s 0% 844k
> 3680+0 records in
> 3680+0 records out
> 1884160 bytes transferred in 512.856684 secs (3674 bytes/sec)
> ^C3680+0 records in
> 3680+0 records out
> 1884160 bytes transferred in 513.392858 secs (3670 bytes/sec)
> Killed by signal 2.
>
> Again, not even 4 KB/s throughput over ssh. What's weird is that not all
> network traffic is slow. I'm able to download at almost my full DSL speed
> (7168 Kbps) and here's the strange bit: NFS is fast! From the same
> machine as above, I have an NFS mountpoint:
>
> % dd if=/nfs/some_large_file of=/dev/null
> 55980+0 records in
> 55980+0 records out
> 28661760 bytes transferred in 0.147159 secs (194767476 bytes/sec)
>
> Which is what I would expect for a gigabit network. Even if I perform the
> same test over ssh:
>
> % ssh workstation cat /nfs/some_large_file | dd of=/dev/null
> 55980+0 records in
> 55980+0 records out
> 28661760 bytes transferred in 2.791392 secs (10267909 bytes/sec)
>
> This is reasonable considering ssh overhead. The kernel config for the
> server contains:
>
> include GENERIC
> ident DDB
> options KDB
> options KDB_TRACE
> options DDB
> options DDB_NUMSYM
>
> My internal network is on 172.23.20.x and the DSL modem is connected to the
> same NIC on 192.168.0.1:
>
> # ifconfig nfe0
> nfe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=10b<RXCSUM,TXCSUM,VLAN_MTU,TSO4>
> ether 00:15:f2:17:0c:20
> inet 172.23.20.1 netmask 0xffffff00 broadcast 172.23.20.255
> inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255
> media: Ethernet autoselect (1000baseTX <full-duplex,flag0,flag1>)
> status: active
>
> # ipfw show
> 00100 62968 8170100 allow ip from any to any via lo0
> 00110 0 0 deny ip from any to 127.0.0.0/8
> 00120 0 0 deny ip from 127.0.0.0/8 to any
> 00130 25444031 23419621364 divert 8668 ip from any to any via 192.168.0.3
> 00140 0 0 deny ip from not 172.23.20.0/24 to 172.23.20.0/24 dst-port 137-149,445
> 00150 20639141 15067620538 allow tcp from any to any established
> 00160 21 10683 allow ip from any to any frag
> 00170 79791 4696696 allow tcp from any to any setup
> 65530 5713367 8864760207 allow ip from any to any
> 65535 0 0 deny ip from any to any
>
> Other than that, I'm not doing anything out of the ordinary. Why is NFS
> behaving correctly and why are ssh/smbd connections so slow? I've pasted
> my dmesg output below. I've used this configuration for years and it
> wasn't until a recent RELENG_7 upgrade that I've had any problems. The box
> was 99-100% idle during those tests and I don't see an interrupt storm or
> anything funny like that. Any ideas?
>
> -- Rick C. Petty
>
>
> >From /var/run/dmesg.boot:
>
>
> Copyright (c) 1992-2008 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 7.0-STABLE #5: Sat Aug 2 19:43:25 CDT 2008
> rick at gateway:/usr/obj/usr/src/sys/DDB
> module_register: module sk/miibus already exists!
> Module sk/miibus failed to register: 17
> module_register: module skc/sk already exists!
> Module skc/sk failed to register: 17
> module_register: module pci/skc already exists!
> Module pci/skc failed to register: 17
> module_register: module rl/miibus already exists!
> Module rl/miibus failed to register: 17
> module_register: module cardbus/rl already exists!
> Module cardbus/rl failed to register: 17
> module_register: module pci/rl already exists!
> Module pci/rl failed to register: 17
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4400+ (2211.34-MHz 686-class CPU)
> Origin = "AuthenticAMD" Id = 0x20f32 Stepping = 2
> Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
> Features2=0x1<SSE3>
> AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow!>
> AMD Features2=0x3<LAHF,CMP>
> Cores per package: 2
> real memory = 1073676288 (1023 MB)
> avail memory = 1037037568 (988 MB)
> ACPI APIC Table: <Nvidia AWRDACPI>
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> cpu0 (BSP): APIC ID: 0
> cpu1 (AP): APIC ID: 1
> ioapic0: Changing APIC ID to 2
> ioapic0 <Version 1.1> irqs 0-23 on motherboard
> kbd1 at kbdmux0
> ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
> acpi0: <Nvidia AWRDACPI> on motherboard
> acpi0: [ITHREAD]
> acpi0: Power Button (fixed)
> acpi0: reservation of 0, a0000 (3) failed
> acpi0: reservation of 100000, 3fef0000 (3) failed
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> pci0: <memory> at device 0.0 (no driver attached)
> isab0: <PCI-ISA bridge> at device 1.0 on pci0
> isa0: <ISA bus> on isab0
> pci0: <serial bus, SMBus> at device 1.1 (no driver attached)
> atapci0: <nVidia nForce CK804 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 6.0 on pci0
> ata0: <ATA channel 0> on atapci0
> ata0: [ITHREAD]
> ata1: <ATA channel 1> on atapci0
> ata1: [ITHREAD]
> atapci1: <nVidia nForce CK804 SATA300 controller> port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xd800-0xd80f mem 0xd0002000-0xd0002fff irq 20 at device 7.0 on pci0
> atapci1: [ITHREAD]
> ata2: <ATA channel 0> on atapci1
> ata2: [ITHREAD]
> ata3: <ATA channel 1> on atapci1
> ata3: [ITHREAD]
> atapci2: <nVidia nForce CK804 SATA300 controller> port 0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xc400-0xc40f mem 0xd0001000-0xd0001fff irq 21 at device 8.0 on pci0
> atapci2: [ITHREAD]
> ata4: <ATA channel 0> on atapci2
> ata4: [ITHREAD]
> ata5: <ATA channel 1> on atapci2
> ata5: [ITHREAD]
> pcib1: <ACPI PCI-PCI bridge> at device 9.0 on pci0
> pci5: <ACPI PCI bus> on pcib1
> vgapci0: <VGA-compatible display> mem 0xc0000000-0xc7ffffff irq 16 at device 6.0 on pci5
> atapci3: <Promise PDC40718 SATA300 controller> port 0xa000-0xa07f,0xa400-0xa4ff mem 0xc9041000-0xc9041fff,0xc9000000-0xc901ffff irq 17 at device 7.0 on pci5
> atapci3: [ITHREAD]
> atapci3: [ITHREAD]
> ata6: <ATA channel 0> on atapci3
> ata6: [ITHREAD]
> ata7: <ATA channel 1> on atapci3
> ata7: [ITHREAD]
> ata8: <ATA channel 2> on atapci3
> ata8: [ITHREAD]
> ata9: <ATA channel 3> on atapci3
> ata9: [ITHREAD]
> atapci4: <Promise PDC40718 SATA300 controller> port 0xa800-0xa87f,0xac00-0xacff mem 0xc9040000-0xc9040fff,0xc9020000-0xc903ffff irq 18 at device 8.0 on pci5
> atapci4: [ITHREAD]
> atapci4: [ITHREAD]
> ata10: <ATA channel 0> on atapci4
> ata10: [ITHREAD]
> ata11: <ATA channel 1> on atapci4
> ata11: [ITHREAD]
> ata12: <ATA channel 2> on atapci4
> ata12: [ITHREAD]
> ata13: <ATA channel 3> on atapci4
> ata13: [ITHREAD]
> nfe0: <NVIDIA nForce4 CK804 MCP9 Networking Adapter> port 0xb000-0xb007 mem 0xd0000000-0xd0000fff irq 22 at device 10.0 on pci0
> miibus0: <MII bus> on nfe0
> e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 9 on miibus0
> e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
> nfe0: Ethernet address: 00:15:f2:17:0c:20
> nfe0: [FILTER]
> pcib2: <ACPI PCI-PCI bridge> at device 11.0 on pci0
> pci4: <ACPI PCI bus> on pcib2
> pcib3: <ACPI PCI-PCI bridge> at device 12.0 on pci0
> pci3: <ACPI PCI bus> on pcib3
> pcib4: <ACPI PCI-PCI bridge> at device 13.0 on pci0
> pci2: <ACPI PCI bus> on pcib4
> pcib5: <ACPI PCI-PCI bridge> at device 14.0 on pci0
> pci1: <ACPI PCI bus> on pcib5
> cpu0: <ACPI CPU> on acpi0
> powernow0: <Cool`n'Quiet K8> on cpu0
> device_attach: powernow0 attach returned 6
> cpu1: <ACPI CPU> on acpi0
> powernow1: <Cool`n'Quiet K8> on cpu1
> device_attach: powernow1 attach returned 6
> acpi_button0: <Power Button> on acpi0
> acpi_tz0: <Thermal Zone> on acpi0
> pmtimer0 on isa0
> orm0: <ISA Option ROM> at iomem 0xc0000-0xccfff pnpid ORM0000 on isa0
> atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> atkbd0: [ITHREAD]
> ppc0: parallel port not found.
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> sio0: configured irq 4 not in bitmap of probed irqs 0
> sio0: port may not be enabled
> sio0: configured irq 4 not in bitmap of probed irqs 0
> sio0: port may not be enabled
> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
> sio0: type 8250 or not responding
> sio0: [FILTER]
> sio1: configured irq 3 not in bitmap of probed irqs 0
> sio1: port may not be enabled
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> Timecounters tick every 1.000 msec
> ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based forwarding disabled, default to deny, logging disabled
> ad0: 157066MB <Hitachi HDS722516VLAT80 V34OA6MA> at ata0-master UDMA100
> ad12: 715404MB <Seagate ST3750330AS SD04> at ata6-master SATA150
> ad14: 715404MB <Seagate ST3750330AS SD04> at ata7-master SATA150
> ad16: 715404MB <Seagate ST3750330AS SD04> at ata8-master SATA150
> ad18: 715404MB <Seagate ST3750330AS SD04> at ata9-master SATA150
> ad20: 476940MB <WDC WD5000AAKS-65TMA0 12.01C01> at ata10-master SATA300
> ad22: 476940MB <WDC WD5000AAKS-65TMA0 12.01C01> at ata11-master SATA300
> ad24: 476940MB <WDC WD5000AAKS-00TMA0 12.01C01> at ata12-master SATA300
> ad26: 476940MB <WDC WD5000AAKS-65TMA0 12.01C01> at ata13-master SATA300
> SMP: AP CPU #1 Launched!
> GEOM_LABEL: Label for provider ad0s1g is ufs/cache.
> Trying to mount root from ufs:/dev/ad0s1a
> GEOM_LABEL: Label ufs/cache removed.
> GEOM_LABEL: Label for provider ad0s1g is ufs/cache.
> GEOM_LABEL: Label ufs/cache removed.
> nfe0: link state changed to UP
>
> Additional messages from dmesg -a:
>
> Aug 16 14:02:20 gateway login: ROOT LOGIN (root) ON ttyv3
> nfe0: promiscuous mode enabled
> nfe0: promiscuous mode disabled
i immediatly think dns issues if samba and ssh are slow. are your
hosts reverse lookup-able?
regards,
usleep
More information about the freebsd-hackers
mailing list