Abysmally slow write to geom class volume over network

[Michael Osipov]

Am 2016-10-13 um 03:41 schrieb John-Mark Gurney:
> Michael Osipov wrote this message on Wed, Oct 12, 2016 at 20:54 +0200:
>> As if there is a bottleneck between socket read and geom write to FS.
>>
>> Is that better?
>
> Have you run gstat on the system to see if there is an IO bottle
> neck?  Since you are using graid3, you want to look to see if
> it's %busy is ~100, while the underlying components are not.

This is hardly impossible because as soon as I start some SFTP transfer, 
all of my SSH sessions free or receive connetion timeout/abort.  Doing a 
SFTP from FreeBSD to FreeBSD gives me on both physical disks and RAID3 
volume a busy of zero to one perfect. In other terms, the drives are bored.

> Also, if you do dd if=/dev/zero of=/mnt/somefile bs=1m count=1024,
> from the local system, are things still slow?

I have done this in the first place, this is fast:

# dd if=/dev/zero bs=1M count=10240 | pv -s 10G -treab --progress > 
/mnt/file.bin
10240+0 records out
10737418240 bytes transferred in 216.084333 secs (49690869 bytes/sec)
   10GiB 0:03:36 [47.4MiB/s] [47.4MiB/s] [==========================>] 100%


> P.S. This is basic debugging, have to identify which part of the
> system is misbehaving.

I am aware of that. Thank you.

> P.P.S. Your "non-geom" test is still using geom, just not w/ the graid3
> module.

Oh ok, I wasn't aware of that. Actually, I have tried all of them: 
concat, stripe, mirror, gvinum. Local operations are fast, though remote 
still crawls slowly.

Michael