gmirror RAID-1: rebuilding freezes machine

[Pawel Jakub Dawidek]

On Sat, Sep 09, 2006 at 04:26:52PM +0200, Volker wrote:
> One feature request on gmirror:
> 
> If a system comes up and more than one mirror is out of sync,
> currently gmirror tries (in automatic mode) to re-sync all providers
> at the same time which slows down the system.
> 
> When gmirror tries to do a resync, it would make sense to do that
> one by one and not all at the same time as the disk head moves
> erratically and seek time is wasted time. Because of that I'm now
> using automatic rebuild mode on the first (root-fs) gmirror slice
> and will fire a manual re-sync on the others one by one if needed. I
> guess I'll put that logic in an rc script but gmirror might avoid
> doing concurrent re-syncs and instead using a queue like behavior.

This is quite complex issue to solve:

- If you have 8 disks in your server, every two are in mirror
  configuration, you do want to synchronize all the mirrors in parallel.
- If you have 2 disks and 4 mirrors configured on partitions from those
  disks, you want to synchronize them one by one.
- If you have UFS file system on a mirror provider, you want to delay
  synchronization or delay fsck, but don't want them to run together.
- Hmm, maybe you have also raid3 configured on the same disks you have
  mirrors?

Basically every mirror is a separate entity and doesn't have any
knowledge about the rest of mirrors (or any other GEOM classes).
Keeping it that way is a good thing, IMHO.
Another thing is that GEOM as an infrastructure doesn't want me to treat
some providers as special, ie. I shouldn't care if mirror is configured
on plain disks or on partitions. And maybe one component is a disk and
one is a partition (it could be useful when one has disks of different
sizes).

Taking all of this into account, I don't think such logic should be
implemented in gmirror itself. I do agree that we should provide some
userland (rcNG? daemon?) infrastructure to allow to easly configure
order of gmirror synchronization, graid3 synchronization, fsck, gvinum
synchronization(?), etc.

For example we have those providers:
mirror/foo1 (ad0s1 ad1s1)
mirror/foo2 (ad0s2 ad1s2)
mirror/foo3 (ad0s3 ad1s3)
mirror/bar (ad2s1 ad3s1)
raid3/baz (ad0s4 ad1s4 ad2s2)

We have a daemon (shell script?) runing heavy disk tasks in the given
order:

	fsck:mirror/foo1
	fsck:mirror/bar
	fsck:mirror/foo2 (after fsck:mirror/foo1)
	fsck:mirror/foo3 (after fsck:mirror/foo2)
	fsck:raid3/baz (after fsck:mirror/foo2,fsck:mirror/bar)
	sync:mirror/foo1 (after fsck:raid3/baz)
	sync:mirror/bar (after fsck:raid3/baz)
	sync:mirror/foo2 (after sync:mirror/foo1)
	sync:mirror/foo3 (after sync:mirror/foo2)
	sync:raid3/baz (after sync:mirror/foo2,sync:mirror/bar)

I hope this is easy.

I'm also planning to implement some messages which will be passed via
devd(8), like 'component X disconnected from Y', 'synchronization of
component X in Y finished', etc. which could be helpful for such a
daemon.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20060909/7c6661c5/attachment.pgp