My experies with gvirstor
Patrick Tracanelli
eksffa at freebsdbrasil.com.br
Sun Apr 29 20:10:36 UTC 2007
Here is what I got as my experiences with gvirstor so far.
With kern.geom.virstor.debug=15 get really slow. While accessing
(newfs'ing) /dev/virstor/home system starts to get 98% of CPU cycles. No
problem after all, just mentioning in case this shouldnt happen.
With a 40GB device I cannot create a 2TB virtual gvirstor device. While
in newfs, in a certain moment I get:
...
...
...
3542977888, 3543354240, 3543730592, 3544106944, 3544483296, 3544859648,
3545236000, 3545612352, 3545988704, 3546365056,
3546741408, 3547117760, 3547494112,
and it STOPS. Checking for debug I can find out that BIO Delaying is
working, because I get:
GEOM_VIRSTOR[1]: All physical space allocated for home
GEOM_VIRSTOR[2]: Delaying BIO (size=65536) until free physical space can
be found on virstor/home
If I ./gvirstor add home ad4s1, things start to work back. But ad4s1 is
way too small and Is not enough. But at least a 1G device I can create:
/dev/virstor/home 946G 4.0K 870G 0% /usr/home4
So my question, how can I make the math to find out how much real space
I will need to create a gvirstor device sized N?
# ./gvirstor status home
Name Status Components
virstor/home 43% physical free ad2s1
Since it is a 40GB devie, something close to 34GB was used to store
structure of a 1TB device. Is this usage related to the chunk size?
Anyway I still adding and removing disks from the gvirstor
Name Status Components
virstor/home 91% physical free ad2s1
ad0s2d
ggate0
I am also importing ggate devices into gvirstor wihtout problems. It is
amazing that it works since it gives us teorically ilimited storage
space. In a single machine we are limited at number of disks. But with
multiple machines importing ggate exports, it seems amazingly.
however, if I export gvirstor device, the other side (ggate client) can
only import it if it is umounted in the local machine (the one where
gvirstor resides):
/dev/ggate0 946G 4.0K 870G 0% /mnt
If I try to mount it, I get:
# mount /dev/virstor/home /usr/home4
mount: /dev/virstor/home: Operation not permitted
And, if I try to export a mounted gvirstor device, ggated starts fine:
# ggated -v
info: Reading exports file (/etc/gg.exports).
debug: Added 10.69.69.69/32 /dev/md0 RW to exports list.
debug: Added 10.69.69.69/32 /dev/virstor/home RW to exports list.
info: Exporting 2 object(s).
info: Listen on port: 3080.
But on remote side:
# ggatec create -v -o rw 10.69.69.1 /dev/virstor/home
info: Connected to the server: 10.69.69.1:3080.
debug: Sending version packet.
debug: Sending initial packet.
debug: Receiving initial packet.
debug: Received initial packet.
info: Connected to the server: 10.69.69.1:3080.
debug: Sending version packet.
debug: Sending initial packet.
debug: Receiving initial packet.
debug: Received initial packet.
error: ggatec: ioctl(/dev/ggctl): Invalid argument.
And, back on the server side, ggated:
error: Cannot open /dev/virstor/home: Operation not permitted.
debug: Connection removed [10.69.69.69 /dev/virstor/home].
warning: Cannot send initial packet: Bad file descriptor.
That´s bad fun :( I thought I could do more lego play. This seems like
the same problem I had in the past, trying to export a mounted gmirror
device.
Its probably a ggate limitation.
Anyways, this is again, just to report the experiences I had by now.
Up to now I didnt get panics or unexpected behavior.
With this command:
# dd if=/dev/zero of=/usr/home4/40G.bin count=40000 bs=1024k
While running
iostat -w1 ad0 ad2
I can see there is no performance difference comparing writings to the
ad provider or to a gvirstor provider. I can also see that the disk
usage is one provider each time. I only get activiry on ad0 when ad2 has
ended up its space. gstat shows me the same thing.
However, let me ask something. Is metadata information updated
synchronously?
I ask it because removing /usr/home4/40G.bin (rm /usr/home4/40G.bin)
takes about 1 and a half minute to finish (newfs was made with -U flag).
I will do some more testings during the week.
More information about the freebsd-current
mailing list