Creating zpool on NVMe Disks takes forever

[Steven Hartland]

On 19/02/2015 13:02, Tom Evans wrote:
> On Thu, Feb 19, 2015 at 12:22 PM, Michael Fuckner <michael at fuckner.net> wrote:
>> On 02/19/2015 12:56 PM, Steven Hartland wrote:
>>> Disable trim on init:
>>> sysctl vfs.zfs.vdev.trim_on_init=0
>> this fixed it, thx!
>>
>> but why does it take >8h to trim 2x 400GB? Or is trimming handled
>> differently on NVMe than on HDD/SSD?
> TRIM/UNMAP is simply an SATA/SCSI command that is sent to a device.
> What the device does when it gets that command is up to the controller
> and firmware on the device.
>
Technically is not TRIM / UNMAP its BIO_DELETE which is translated by 
the relevant device driver to the format the device understands.

For SCSI this can be TRIM, UNMAP, WS or ZERO, for ATA this can be 
CFA_ERASE or TRIM and for NVMe this is a Deallocate.

One reason why it might be slow is I don't see NVMe configuring geom 
d_delmaxsize, which if the case will force the "TRIM" to be split into 
single sector deallocation requests.

     Regards
     Steve