NFS on 10G interface terribly slow

[Rick Macklem]

Gerrit Kuhn wrote:
> On Fri, 26 Jun 2015 20:42:08 -0400 (EDT) Rick Macklem
> <rmacklem at uoguelph.ca> wrote about Re: NFS on 10G interface terribly slow:
> 
> RM> Btw, can you tell us what Intel chip(s) you're using?
> 
> I have
> 
> ix0 at pci0:5:0:0: class=0x020000 card=0x00028086 chip=0x15288086 rev=0x01
> hdr=0x00 vendor     = 'Intel Corporation'
>     device     = 'Ethernet Controller 10-Gigabit X540-AT2'
>     class      = network
>     subclass   = ethernet
> 
> RM> For example, from the "ix" driver:
> RM> #define IXGBE_82598_SCATTER		100
> RM> #define IXGBE_82599_SCATTER		32
> 
> Hm, I cannot find out into which chipset number this translates for my
> device...
> 
> RM> Btw, it appears that the driver in head/current now sets
> RM> if_hw_tsomaxsegcount, but the driver in stable/10 does not. This means
> RM> that the 82599 chip will end up doing the m_defrag() calls for 10.x.
> 
> So the next step could even be updating to -current...
> OTOH, I get the same (bad) resulsts, no matter if TSO is enabled or
> disabled on the interface.
> 
Since disabling TSO had no effect, I don't think updating would matter.

If you can test against a different NFS server, that might indicate whether
or not the Solaris server is the bottleneck.

If the Solaris server is using ZFS, setting sync=disabled might help w.r.t.
write performance. It is, however, somewhat dangerous w.r.t. loss of recently
written data when the server crashes. (Server has told client data is safely
on stable storage so client will not re-write the block(s) although data wasn't
on stable storage and is lost.)
(I'm not a ZFS guy, so I can't suggest more w.r.t. ZFS.)

rick

> 
> cu
>   Gerrit
>