Gmirror performanc

[Michel Talon]

Oliver Fromme <olli at lurza.secnetix.de> wrote:

>  > I tried with -b split -s <various sizes>, -b round-robin, -b load.
>  > (dd-ing as done with a bs of 1m; I see the transaction size is 128Kb,
>  > unless the split method is used, in which case the transaction size
>  > gies down. When round-robin is used, the transaction size is 128Kb/s,
>  > but the number of transaction per second goes down.).
>  > 
>  > I cannot explain why I should not get a higher read speed. Anyone?
> 
> dd is a sequential, single-threaded operation, so it will
> only use one disk at a time.  It's not really suitable as
> a benchmark for real-world things.

I see the same problem as Guido, that is gmirror on two disks is not faster
that each disk separately and is even markedly slower. I disagree with your
explanation, which moreover contradicts the definition of split, round-robin,
etc. in the man page of gmirror. Experimentally, observing the disk throughput
via iostat shows that both disks are involved in the IO.
asmodee# dd if=/dev/mirror/gms1a of=/dev/null bs=256k
556+0 records in
556+0 records out
145752064 bytes transferred in 8.785839 secs (16589431 bytes/sec)
(note the low throughput, the disks are individually able to do more than 20
MB/s, this is on an old machine), while iostat shows:
asmodee% iostat -w 1
      tty             ad0              ad4             cpu
 tin tout  KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
   0   60 61.38 116  6.95  61.40 117  7.01   0  0  1  0 99
   0   60 64.00 129  8.06  64.00 128  7.99   0  0  2  1 97
   0   60 64.00 128  7.99  64.00 129  8.06   0  0  2  1 97
   0   60 64.00 129  8.06  64.00 129  8.06   0  0  1  1 98
   0   60 63.50 128  7.93  63.50 127  7.87   0  0  5  2 94
that is the transaction is evenly distributed on both disks ad0 and ad4
(which are on 2 separate channels). The problem is that each disk works at
only 8MB/s while it is able of 3 times more.

Looking at the kernel driver /usr/src/sys/geom/mirror/g_mirror.c, it seems
that the load is split on the various disks in the function
g_mirror_request_split() in a way which is simulatneous for all providers.
How is it that after that the request proceeds so slowly, i don't know. But
i doubt very much it will be any different wether you have a "real world load" 
or a simple dd.


-- 

Michel TALON