Re: Upsizing a ZFS filesystem - shift question

From: Daniel Kalchev <daniel_at_digsys.bg>
Date: Fri, 28 May 2021 14:50:13 UTC
The ashift property is on a per-vdev — so you can have different ashift vdevs in the same pool.

Another best bet is, if the system is running recent FreeBSD with ZFS that has the device_removal feature and you are not using raidz.

You can add a new vdev, then remove the old vdevs one by one. Eventually, you will end up with the new vdevs only.

This will ensure data is online at all time, but performance will be of course best with recreating the zpool — you can use zfs send/receive to move data to the new pool.

Daniel

> On 27 May 2021, at 21:36, Alan Somers <asomers@freebsd.org> wrote:
> 
> ashift affects how the data is laid out on disk, so you can't change it
> after the drives have been replaced.  And you don't want to use ashift=9
> for the new drives; their performance will suck.  Your best bet is to
> create a new zpool, then send | receive the data from the old pool.
> -Alan
> 
> On Thu, May 27, 2021 at 12:33 PM joe mcguckin <joe@via.net> wrote:
> 
>> I have an existing 32TB filesystem (2 X 8T vdevs). I want to upgrade each
>> of the drives to 16TB. I pulled one drive and tried a ‘zfs replace’ comand,
>> but zfs replied with some message about new drive
>> optimal ashift doesn’t match the vdev. It suggested Retrying  the
>> ‘replace’ command with ‘zfs replace -o ashift=N’
>> 
>> Ok, a little investigation shows the existing filesystem has an ashift of
>> 9. These are older 2T drives with 512 byte sectors. The newer drives are
>> 4kn sectors.
>> 
>> Is ashift settable on a drive by drive basis? Can I temporarily set ashift
>> on a drive until all the drives in a vdev have been replaced?  Once all
>> drives have been replaced, how do I set ashift=12 for all the drives in the
>> vdev?
>> 
>> After replacing all the drives, will the additional space magicially
>> appear or is there an additional command or series of steps?
>> 
>> Thanks,
>> 
>> Joe
>> 
>> Joe McGuckin
>> ViaNet Communications
>> 
>> joe@via.net
>> 650-207-0372 cell
>> 650-213-1302 office
>> 650-969-2124 fax
>> 
>> 
>> 
>>