GEOM mirror performance - slow

[Ahnjoan Amous]

I'm sorry about the double posting, I have switch e-mail clients and
hope that the text is now understandable.

While deploying 6.1-RELEASE I'm seeing slower than expected speed
using geom mirror.

Test 1.  Outside of GEOM I'm able to write to both aacd2 and
aacd3 concurrently and achieve between 41M – 45M a second write
speeds. (around 75M a second if I run the commands separately)

Test 2. Inside a GEOM container but not mirrored (aacd2 = gm2,
aacd3 = gm3) I'm able to write to both gm2 and gm3 concurrently
and achieve between 40M – 49M a second write speeds. (around 75M
a second if I run the commands separately)

Test 3. Inside a GEOM mirror (gm1 = aacd2 = aacd3) I'm able to
achieve 23M a second.

I'd love someones opinion as to if the speed I'm seeing using
GEOM mirror is to be expected.

FYI – The mirror was manually rebuilt and COMPLETE before tests
were attempted on the mirror.

***********************************************************************
*** Test 1.
*** dd to devices mounted outside of geom

> dd if=/dev/zero of=/data02/helloworld bs=8k count=100k &
> dd if=/dev/zero of=/data03/helloworld bs=8k count=100k &
   838860800 bytes transferred in 18.799345 secs (44621810 bytes/sec)
                                                  ^^^^^^^^
   838860800 bytes transferred in 18.544963 secs (45233889 bytes/sec)
                                                  ^^^^^^^^
> dd if=/dev/zero of=/data02/helloworld bs=1m count=800 &
> dd if=/dev/zero of=/data03/helloworld bs=1m count=800 &
   838860800 bytes transferred in 20.014247 secs (41913183 bytes/sec)
                                                  ^^^^^^^^
   838860800 bytes transferred in 19.824656 secs (42314016 bytes/sec)
                                                  ^^^^^^^^

   **** setup info for test ****
   > df -k
       /dev/aacd2s1e   2026030 819620 1044328    44%    /data02
       /dev/aacd3s1e   2026030 819620 1044328    44%    /data03

***********************************************************************
*** Test 2.
*** dd to devices mounted as their own geom volumes

> dd if=/dev/zero of=/data02/helloworld bs=8k count=100k &
> dd if=/dev/zero of=/data03/helloworld bs=8k count=100k &
   838860800 bytes transferred in 17.118679 secs (49002659 bytes/sec)
                                                  ^^^^^^^^
   838860800 bytes transferred in 16.810787 secs (49900150 bytes/sec)
                                                  ^^^^^^^^
> dd if=/dev/zero of=/data02/helloworld bs=1m count=800 &
> dd if=/dev/zero of=/data03/helloworld bs=1m count=800 &
   838860800 bytes transferred in 20.907455 secs (40122569 bytes/sec)
                                                  ^^^^^^^^
   838860800 bytes transferred in 20.638277 secs (40645874 bytes/sec)
                                                  ^^^^^^^^

   **** setup info for test ****
   > df -k
       /dev/mirror/gm2   2026030 819620 1044328    44%    /data02
       /dev/mirror/gm3   2026030 819620 1044328    44%    /data03
   > gmirror list
       Geom name: gm2
         1. Name: aacd2s1e
       Geom name: gm3
         1. Name: aacd3s1e

***********************************************************************
*** Test 3.
*** dd to devices mounted as a geom mirror

> dd if=/dev/zero of=/data01/helloworld bs=8k count=100k
   838860800 bytes transferred in 35.622392 secs (23548694 bytes/sec)
                                                  ^^^^^^^^
> dd if=/dev/zero of=/data01/helloworld bs=1m count=800
   838860800 bytes transferred in 35.588491 secs (23571126 bytes/sec)
                                                  ^^^^^^^^

   **** setup info for test ****
   > df -k
       /dev/mirror/gm1   2026028 819620 1044326    44%    /data01
   > gmirror list
       Geom name: gm1
         1. Name: aacd2s1e
         2. Name: aacd3s1e