zfs send/receive: is this slow?

[Dan Langille]

FYI: this is all on the same box. 

-- 
Dan Langille
http://langille.org/


On Oct 1, 2010, at 5:56 PM, Artem Belevich <fbsdlist at src.cx> wrote:

> Hmm. It did help me a lot when I was replicating ~2TB worth of data
> over GigE. Without mbuffer things were roughly in the ballpark of your
> numbers. With mbuffer I've got around 100MB/s.
> 
> Assuming that you have two boxes connected via ethernet, it would be
> good to check that nobody generates PAUSE frames. Some time back I've
> discovered that el-cheapo switch I've been using for some reason could
> not keep up with traffic bursts and generated tons of PAUSE frames
> that severely limited throughput.
> 
> If you're using Intel adapters, check xon/xoff counters in "sysctl
> dev.em.0.mac_stats". If you see them increasing, that may explain slow
> speed.
> If you have a switch between your boxes, try bypassing it and connect
> boxes directly.
> 
> --Artem
> 
> 
> 
> On Fri, Oct 1, 2010 at 11:51 AM, Dan Langille <dan at langille.org> wrote:
>> 
>> On Wed, September 29, 2010 2:04 pm, Dan Langille wrote:
>>> $ zpool iostat 10
>>>                capacity     operations    bandwidth
>>> pool         used  avail   read  write   read  write
>>> ----------  -----  -----  -----  -----  -----  -----
>>> storage     7.67T  5.02T    358     38  43.1M  1.96M
>>> storage     7.67T  5.02T    317    475  39.4M  30.9M
>>> storage     7.67T  5.02T    357    533  44.3M  34.4M
>>> storage     7.67T  5.02T    371    556  46.0M  35.8M
>>> storage     7.67T  5.02T    313    521  38.9M  28.7M
>>> storage     7.67T  5.02T    309    457  38.4M  30.4M
>>> storage     7.67T  5.02T    388    589  48.2M  37.8M
>>> storage     7.67T  5.02T    377    581  46.8M  36.5M
>>> storage     7.67T  5.02T    310    559  38.4M  30.4M
>>> storage     7.67T  5.02T    430    611  53.4M  41.3M
>> 
>> Now that I'm using mbuffer:
>> 
>> $ zpool iostat 10
>>               capacity     operations    bandwidth
>> pool         used  avail   read  write   read  write
>> ----------  -----  -----  -----  -----  -----  -----
>> storage     9.96T  2.73T  2.01K    131   151M  6.72M
>> storage     9.96T  2.73T    615    515  76.3M  33.5M
>> storage     9.96T  2.73T    360    492  44.7M  33.7M
>> storage     9.96T  2.73T    388    554  48.3M  38.4M
>> storage     9.96T  2.73T    403    562  50.1M  39.6M
>> storage     9.96T  2.73T    313    468  38.9M  28.0M
>> storage     9.96T  2.73T    462    677  57.3M  22.4M
>> storage     9.96T  2.73T    383    581  47.5M  21.6M
>> storage     9.96T  2.72T    142    571  17.7M  15.4M
>> storage     9.96T  2.72T     80    598  10.0M  18.8M
>> storage     9.96T  2.72T    718    503  89.1M  13.6M
>> storage     9.96T  2.72T    594    517  73.8M  14.1M
>> storage     9.96T  2.72T    367    528  45.6M  15.1M
>> storage     9.96T  2.72T    338    520  41.9M  16.4M
>> storage     9.96T  2.72T    348    499  43.3M  21.5M
>> storage     9.96T  2.72T    398    553  49.4M  14.4M
>> storage     9.96T  2.72T    346    481  43.0M  6.78M
>> 
>> If anything, it's slower.
>> 
>> The above was without -s 128.  The following used that setting:
>> 
>>  $ zpool iostat 10
>>               capacity     operations    bandwidth
>> pool         used  avail   read  write   read  write
>> ----------  -----  -----  -----  -----  -----  -----
>> storage     9.78T  2.91T  1.98K    137   149M  6.92M
>> storage     9.78T  2.91T    761    577  94.4M  42.6M
>> storage     9.78T  2.91T    462    411  57.4M  24.6M
>> storage     9.78T  2.91T    492    497  61.1M  27.6M
>> storage     9.78T  2.91T    632    446  78.5M  22.5M
>> storage     9.78T  2.91T    554    414  68.7M  21.8M
>> storage     9.78T  2.91T    459    434  57.0M  31.4M
>> storage     9.78T  2.91T    398    570  49.4M  32.7M
>> storage     9.78T  2.91T    338    495  41.9M  26.5M
>> storage     9.78T  2.91T    358    526  44.5M  33.3M
>> storage     9.78T  2.91T    385    555  47.8M  39.8M
>> storage     9.78T  2.91T    271    453  33.6M  23.3M
>> storage     9.78T  2.91T    270    456  33.5M  28.8M
>> 
>> 
>> _______________________________________________
>> freebsd-stable at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>> 
>