ZFS on 4K drives - Expanding zpool and adding spares (gnop method)

Alexander Leidinger Alexander at Leidinger.net
Wed Jun 1 08:19:42 UTC 2011

Quoting Negasi Gerima <negasi at decayingorbits.com> (from Wed, 01 Jun  
2011 02:09:53 -0400):

> Hello,
> I apologize if this topic is redundant, but I couldn't find an exact  
> answer to my specific question. So here goes.
> I recently decided to expand the storage in my home server with a  
> bunch of 1.5Tb Samsung 4K hard drives. Using the guide at the  
> following link I was able to use the "gnop method" in order to align  
> the drives for use with 4K sectors:
> http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector-drives.html
> My question to the list is this, now that my new zpool is created  
> and I've made sure the drives are using the correct layout, how  
> would I go about expanding the zpool in the future with additional  
> drives and/or spares? I emailed the author of the article but  
> received a reply that I found terse at best.
> I'm assuming that I would have to create a new gnop device for each  
> additional hard drive, and then add those gnop devices to the pool.

The 4k-sector part results in a pool specific config setting. I would  
expect that if you add additional drives (no matter if 4k or not),  
that the pool will continue to use 4k sectors. So I would expect that  
you do not need gnop to expend an existing pool, but I have not  
tested/verified this.

You could test this with md(4) devices where you use gnop for the  
initial creation and no gnop during extending it. When it is extended  
you just run
   zdp <poolname) | grep ashift
and verify that it is set to 12.

> For example, if I had a pool called 'datastore' and three additional  
> drives /dev/ad10, /dev/ad12, & /dev/ad14. I'm assuming I should use  
> the following steps to achieve my goal:
> 1. Create the gnop devices /dev/ad10.nop, /dev/ad12.nop, and /dev/ad14.nop,

In general (I'm not sure if it applies to ZFS): If you use ad10pX.nop  
instead, it should work. If you use gnop on the entire disks instead  
of on partitions, you may get in trouble depending on where some  
important metadata is kept (e.g. gpt/gmirror/glabel/... save metadata  
in the last sector, but the last 4k-gnoped-sector is the 4th last  

for how I create a 4k-safe pool.

> 2. Add these devices to the zpool with 'zpool add datastore  
> /dev/ad10.nop /dev/ad12.nop'
> 3. Export the zpool with 'zpool export datastore'
> 4. Destroy gnop devices with 'gnop destroy /dev/da10.nop  
> /dev/da12.nop /dev/da14.nop'
> 5. Re-import the zpool data with 'zpool import datastore'
> Can someone please confirm that this method is correct, or if there  
> is a more streamlined method for achieving the desired goal?

This (with partitions instead of the whole disks) is the safe way to  
do it. Technically it is not necessary to export the pool and destroy  
the gnop devices. The next reboot will not create the gnop devices and  
ZFS will pick up the drives based upon a zfs-meta-data search on all  


There is nothing so easy but that it becomes difficult
when you do it reluctantly.
		-- Publius Terentius Afer (Terence)

http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137

More information about the freebsd-fs mailing list