AF (4096 byte sector) drives: Can you mix/match in a ZFS pool?

[Tom Evans]

On Wed, Oct 12, 2011 at 4:11 PM, Larry Rosenman <ler at lerctr.org> wrote:
> I have a root on ZFS box with 6 drives, all 400G (except one 500G) in a
> pool.
>
> I want to upgrade to 2T or 3T drives, but was wondering if you can mix/match
> while doing the drive by drive
> replacement.
>
> This is on 9.0-BETA3 if that matters.
>
> Thanks!
>

Hi Larry

I'm in a similar position. I have a 2 x 6 x 1.5TB raidz system,
configured a while ago when I wasn't aware enough of 4k sector drives,
and so ZFS is configured to use 512 byte sectors (ashift=9). All of
the drives in it were 512 byte sector drives, until one of them
failed.

At that point, I couldn't lay my hands on a large capacity drive that
still used 512 byte sectors, so I replaced it with a 4k sector drive,
made sure it was aligned correctly, and hoped for the best. The
performance sucks (500MB/s reads -> 150MB/s reads!), but it 'works',
all my data is safe.

The solution is to make sure that all your vdevs, whether they are
backed by disks that have 512 byte or 4k sectors, are created with 4k
sectors (ashift=12). It won't negatively affect your older disks, and
you won't end up in the position I am in, where I need to recreate the
pool to fix the issue (and have 12TB of data with nowhere to put it!)

Cheers

Tom