ZFS pool restructuring and emergency repair

[Bob Friesenhahn]

On Fri, 19 Jun 2015, Quartz wrote:
>
> What are people's responses and recommendations given the following 
> hypothetical situations:
>
> - A server is set up with a pool created a certain way. A couple years later 
> it's determined that the pool configuration wasn't a good choice for the 
> workload and it should be redone. As I understand it, ZFS has no capability 
> to reorganize, remove, or re-type vdevs, so your only option is completely 
> starting over with another whole pool. Is this still true? If so, is there a 
> correct way to copy an entire pool to another set of disks in a way that 
> preserves all the metadata and hierarchical dataset information? (snapshots, 
> noatime, compression, dedupe, quotas, mountpoints, etc). It looks like 'send' 
> and 'receive' might do it, but I'm having trouble finding detailed 
> information on exactly what they copy, how much of a skeleton on the 
> receiving end I need to manually create first, and what breaks if we have a 
> root-on-ZFS setup.

You can use 'send' and 'receive' to send all the data and the metadata 
associated with that data.  I believe that you are correct that 
filesystem properties (like 'compression') are not preserved. 
Snapshots can be sent and preserved and indeed send/receive is based 
on snapshots.  You can recover filesystem properties at the source via 
'zpool history' but then need to filter out only what is needed and do 
any necessary edits to produce a script for the receiving end.

I have a system which sends several filesystems to another system 
every night via send/receive without any problems, and with different 
properties (e.g. read-only) set on the receive filesystem.  I also 
have nightly send from one pool to another on the same system (all for 
redundant backups).

It is very easy to expand/enlarge a pool (given the physical ability 
to do so) but the only way to restructure the existing parts of a pool 
is to create a new one.

> - A server is set up with a pool created a certain way, for the sake of 
> argument let's say it's a raidz-2 comprised of 6x 2TB disks. There's only 
> actually ~1TB of data currently on the server though. Let's say there's a 
> catastrophic emergency where one of the disks needs to be replaced, but the 
> only available spare is an old 500GB. As I understand it, you're basically 
> SOL. Even though a 6x500 (really 4x500) is more than enough to hold 1Tb of 
> data, you can't do anything in this situation since although ZFS can expand a 
> pool to fit larger disks, it can't shrink one under any circumstance. Is my 
> understanding still correct or is there a way around this issue now?

You will still need to supply a replacement disk at least as large as 
the disk being replaced.

If the disks can be assured to be 100% reliable and the pool is not 
very full, then there are tricky ways to create a new pool on the same 
disks that the old pool occupied.  I am not brave enough to do this 
but others have accomplished it successfully.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/