"sharing" hot spares?

[Boris Kochergin]

On 12/07/10 21:24, Charles Sprickman wrote:
> All,
>
> This is probably an odd question...  We're extensively using ZFS on a 
> bunch of 1U boxes and packing all the drive bays full.  This is all 
> SATA, so I would like to have hot spares available, but without 
> migrating everything to new enclosures, I'm not seeing an easy way to 
> deal with this.
>
> Is there anything out there that would allow me to export a few drives 
> from another box as block devices that the other boxes could 
> *temporarily* use as hot spares until someone onsite could physically 
> do a drive swap?
>
> I would also really like to share one drive amongst multiple hosts - I 
> am making the likely over-optimistic assumption that I'll be dealing 
> with no more than one drive failure per say, 3 hosts at a time.
>
> -hast doesn't seem quite right, as it's more about clustering than 
> just exporting a device.
>
> -geom-gate looks like it might fill the bill, but I'm unsure as to 
> whether it would allow an export of one drive to more than one box, 
> and if it did, if it would have any way of letting a second host know 
> the drive is busy.
>
> -iscsi seems like it might work as well, but I'm leery of anything not 
> deemed stable/supported enough to be in the base system.
>
> Anything else?
>
> I'm also totally clueless as to how zfs would deal with having one 
> drive coming over the network while the others are local.  I assume it 
> should work.
>
> Also if there was a reboot while it was running with a disk from this 
> network spare "pool", could zfs then find the network drive it was 
> using since it will be available later than the local drives?
>
> This whole scheme is just something that came to me a few minutes ago, 
> my apologies if it sounds a bit nuts.  Just trying to solve an 
> interesting problem.
>
> Thanks,
>
> Charles
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"

Ahoy. a geom_gate device can indeed be exported to multiple hosts, and 
ZFS over geom_gate works fine. As for synchronization, you don't get any 
with geom_gate itself--you can attach a device for writing from multiple 
hosts. However, I also believe ZFS will not use a disk (unless maybe you 
force it) if it currently has ZFS metadata on it. If this is the case, 
you could leave it to ZFS not to use a spare that's already in use by 
another host. In this scenario, I suppose there might exist a race 
condition where two machines' disks fail at the same time, and they 
attempt to use the geom_gate spare at the same time. I'll leave it to 
you to decide whether to worry about this.

-Boris