RFC: Suggesting ZFS "best practices" in FreeBSD

[Warren Block]

On Tue, 22 Jan 2013, Michael DeMan wrote:

> On Jan 22, 2013, at 7:04 AM, Warren Block <wblock at wonkity.com> wrote:
>>
>> I'm a proponent of using various types of labels, but my impression 
>> after a recent experience was that ZFS metadata was enough to 
>> identify the drives even if they were moved around.  That is, ZFS 
>> bare metadata on a drive with no other partitioning or labels.
>>
>> Is that incorrect?
>
> I don't know if it is correct or not, but the best I could figure out 
> was to both label the drives and also force the mapping so the 
> physical and logical drives always show up associated correctly. I 
> also ended up deciding I wanted the hostname as a prefix for the 
> labels - so if they get moved around to say another machine I can look 
> and know what is going on - 'oh yeah, those disks are from the ones we 
> moved over to this machine'...

It helps to avoid duplicate labels, a good idea.

> #1.  Map the physical drive slots to how they show up in FBSD so if a 
> disk is removed and the machine is rebooted all the disks after that 
> removed one do not have an 'off by one error'.  i.e. if you have 
> ada0-ada14 and remove ada8 then reboot - normally FBSD skips that 
> missing ada8 drive and the next drive (that used to be ada9) is now 
> called ada8 and so on...

How do you do that?  If I'm in that situation, I think I could find the 
bad drive, or at least the good ones, with diskinfo and the drive serial 
number.  One suggestion I saw somewhere was to use disk serial numbers 
for label values.

> #2.  Use gpart+gnop to deal with 4K disk sizes in a standardized way 
> and also to leave a little extra room so if when doing a replacement 
> disk and that disk is a few MB smaller than the original - it all 
> 'just works'.  (All disks are partitioned to a slightly smaller size 
> than their physical capacity).

I've been told (but have not personally verified) that newer versions of 
ZFS actually leaves some unused space at the end of a drive to allow for 
variations in nominally-sized drives.  Don't know how much.