nfs slowdown (RELENG_8)

[Rick Macklem]

Mike Tancsa wrote:
> I havent started the process of narrowing down where the issue might
> be
> or even if there is an issue, but has anyone else seen a situation
> where
> RELENG_8 i386 nfs client write speed have all of a sudden nose dived
> ?
> Doing a simple cat /dev/zero > /mnt/junk used to yield 500Mb and
> above
> easily. Now, its down to 25Mb.  A RELENG_9 client is able to write to
> the same NFS server at 900Mb.  I tested another i386 RELENG_8 server
> with a similar build date and it gets the same slow speed, where as I
> am
> pretty sure the same test gave around 400Mb with an older kernel.
>  Both
> problem boxes are i386 at r267114 (~ June 5th). Both have em nics,
> but
> they are different em NICs. But, I have some other boxes from around
> the
> same time that are just as fast, but with different em nics /
> chipsets
> as well so I am not sure if its related to the em driver or not.
> 
I suggest you try disabling TSO on the nics (or setting wsize=32786
on all mounts). Usually such slowdowns are related to the underlying
network fabric and there are known TSO issues for nics/drivers that
only handle 32 transmit buffers when 64K writes are being used.

rick

> The problematic em nics are
> 
> em1 at pci0:4:0:0: class=0x020000 card=0x35788086 chip=0x10d38086
> rev=0x00
> hdr=0x00
>      vendor     = 'Intel Corporation'
>      device     = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
>      class      = network
>      subclass   = ethernet
>      cap 01[c8] = powerspec 2  supports D0 D3  current D0
>      cap 05[d0] = MSI supports 1 message, 64 bit
>      cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link
>      x1(x1)
>                   speed 2.5(2.5) ASPM disabled(L0s/L1)
>      cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
> and
> 
> em1 at pci0:6:5:0: class=0x020000 card=0x348d8086 chip=0x10768086
> rev=0x05
> hdr=0x00
>      vendor     = 'Intel Corporation'
>      device     = 'Gigabit Ethernet Controller (82541EI)'
>      class      = network
>      subclass   = ethernet
>      bar   [10] = type Memory, range 32, base 0xea020000, size
>      131072,
> enabled
>      bar   [14] = type Memory, range 32, base 0xea000000, size
>      131072,
> enabled
>      bar   [18] = type I/O Port, range 32, base 0x1100, size 64,
>      enabled
>      cap 01[dc] = powerspec 2  supports D0 D3  current D0
>      cap 07[e4] = PCI-X supports 2048 burst read, 1 split transaction
> 
> 
> And ones that are working as expected
> 
> em1 at pci0:14:0:0:        class=0x020000 card=0x109a15d9
> chip=0x109a8086
> rev=0x00 hdr=0x00
>      vendor     = 'Intel Corporation'
>      device     = 'Intel PRO/1000 PL Network Adaptor (82573L)'
>      class      = network
>      subclass   = ethernet
>      bar   [10] = type Memory, range 32, base 0xe8b00000, size
>      131072,
> enabled
>      bar   [18] = type I/O Port, range 32, base 0x6000, size 32,
>      enabled
>      cap 01[c8] = powerspec 2  supports D0 D3  current D0
>      cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1
>      message
>      cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link
>      x1(x1)
>                   speed 2.5(2.5)
> ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
> ecap 0003[140] = Serial 1 003048ffff9c59f1
> 
> and
> 
> em1 at pci0:14:0:0:        class=0x020000 card=0x109a15d9
> chip=0x109a8086
> rev=0x00 hdr=0x00
>      vendor     = 'Intel Corporation'
>      device     = 'Intel PRO/1000 PL Network Adaptor (82573L)'
>      class      = network
>      subclass   = ethernet
>      bar   [10] = type Memory, range 32, base 0xe1100000, size
>      131072,
> enabled
>      bar   [18] = type I/O Port, range 32, base 0x6000, size 32,
>      enabled
>      cap 01[c8] = powerspec 2  supports D0 D3  current D0
>      cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1
>      message
>      cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link
>      x1(x1)
>                   speed 2.5(2.5)
> ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
> ecap 0003[140] = Serial 1 003048ffff905219
> 
> It might not be an issue with the em driver. Not sure at this point,
> but
> wondering if anyone has noticed any other slow downs with NFS on
> RELENG_8 ?
> 
> 
> 	---Mike
> --
> -------------------
> Mike Tancsa, tel +1 519 651 3400
> Sentex Communications, mike at sentex.net
> Providing Internet services since 1994 www.sentex.net
> Cambridge, Ontario Canada   http://www.tancsa.com/
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to
> "freebsd-stable-unsubscribe at freebsd.org"
>