nvd disk on nvme controller not detected at boot-time

[Maikel Verheijen]

Hi Jim,
On Tue, Oct 14, 2014 at 1:47 AM, Maikel Verheijen <Maikel.Verheijen at redwood.com<mailto:Maikel.Verheijen at redwood.com>> wrote:
Hi list!

We recently purchased a HP DL380e G8 server to serve as our backup server with an Intel P3600 that uses the nvme interface. We added load_nvme="YES" and load_nvd="YES" to our loader.conf, and they both get loaded, however the ssd disk device is not detected at boot. When we unload and reload the nvd module the disk does get detected. Is there a way to see if the load-order is correct? We added verbose_loading="YES" to the loader.conf, but dmesg doesn't show me the actual loading.

One thing I did see related to the nvme in the dmesg (full dmesg attached) output is this:


Hi Maikel,

Which version of FreeBSD are you using?  I believe both of these issues have been fixed on HEAD and merged to stable/9 and stable/10.  9.3 and 10.1-RC would have these fixes.  9.2 and 10.0 would not.
I'm currently running FreeBSD 10.0 with updates from freebsd-update. Would FreeBSD 10.1-RC be safe for a production environment or would it be better to wait until the real release? The machine is not used for production yet, but we do want to go live with it in the near future. I currently use a “workaround”, I’m not loading ZFS at boot time but use a custom RC script to load the nvd module, test if the nvd0 disk is there, then load the ZFS module. That seems to work for now, but having it work “out of the box” would be better, and a lot less prone to manual mistakes made by me :)



nvme0: <Generic NVMe Device> mem 0xfbdf0000-0xfbdf3fff irq 16 at device 0.0 on pica
…
nvme0: SET FEATURES (09) sqid:0 cid:9 nsid:0 cdw10:00000080 cdw11:00000000
nvme0: INTERNAL DEVICE ERROR (00/06) sqid:0 cid:9 cdw0:

This was fixed in r263277.  Incidentally, the error message here is harmless and not associated with the primary issue you have described above.
Thanks, this was the only thing I thought that might be relevant to the issue, glad to hear it’s harmless.



After unloading and reloading  the nvd module the disk does get detected:

nvd0: <INTEL SSDPEDME400G4> NVMe namespace
nvd0: 381554MB (781422768 512 byte sectors)

The load on boot issue was fixed in r263310.
That is good news, that means we can probably drop my rc script in the near future and use the proper loading procedures.

Best regards,
-Jim

Thanks again and warm regards,
Maikel Verheijen