NFS reads vs. writes

[Bob Friesenhahn]

On Sat, 2 Jan 2016, Mikhail T. wrote:

> I have two similar 10.2-STABLE machines mounting each other's
> filesystems via NFS: a:/a and b:/b
>
> When moving some large files around today, I found a huge discrepancy
> between writing and reading over NFS. That is, machine a copying files
> from its own /a to NFS-mounted b:/b was pushing measly 2-3Mb/s over the
> Ethernet. When I cancelled that, logged-in to machine b and proceed to
> copy from a:/a to the now-local /b, I got 56Mb/s. I tried changing the
> wsize and rsize options, but it did not seem to make a difference...
>
> Any ideas, what is happening here? Why are NFS-writes some 25 times
> slower here, than reads? Both filesystems are zfs-backed (though with
> different options), both systems are plugged into the same switch, both
> use mtu of 9000.

The most likely issue is a latency problem with synchronous writes on 
'b'.  The main pool disks seem to be working ok.  Make sure that the 
SSD you are using for slog is working fine.  Maybe it is abnormally 
slow.

Check I/O latencies for your disks.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/