Suggestions for working with unstable nvme dev names in AWS

[George Hartzell]

Polytropon writes:
 > On Tue, 14 May 2019 08:59:01 -0700, George Hartzell wrote:
 > > Matthew Seaman writes:
 > >  > [...] but if you
 > >  > are using ZFS, then shuffling the disks around should not make any
 > >  > difference. 
 > >  > [...]
 > > Yes, once I have them set up (ZFS or labeled), it doesn't matter what
 > > device names they end up having.  For now I just do the setup by hand,
 > > poking around a bit.  Same trick in the Linux world, you end up
 > > referring to them by their UUID or ....
 > 
 > In addition to what Matthew suggested, you could use UFS-IDs
 > in case the disks are initialized with UFS. You can find more
 > information here (at the bottom of the page):
 > [...]

Yes.  As I mentioned in my response to Matthew, once I have some sort
of filesystem/zpool on the device, it's straightforward (TMTOWTDI).

The problem is being able to provision the system automatically
without user intervention.

In the Linux world, I can use e.g. Terraform to set up a pair of
additional volumes and tell it to call them `/dev/sdy` and `/dev/sdz`.
The Linux magic happens and I get pair of symlinks that I can use in
my e.g. Ansible playbooks, that point to whatever the devices came up
as when it booted.  I build filesystems on the devices, add them via
their UUID's to `/etc/fstab` and I'm off and running.

I can't [seem to] do this in the FreeBSD world; even if I name the
devices `/dev/nvme1` (the fast and big one) and `/dev/nvme2` (the slow
and small one), there's no guarantee that they'll have those names
when the machine boots.

This is a weirdly AWS issue and their peace offering is to stash the
requested device name in the device/controller/"hardware" and provide
a tool that digs it out.

I'm trying to figure out what I can do about it from FreeBSD.  Perhaps
there's already a solution.  Perhaps the nvme driver needs to be
extended to provide access to the magic AWS info stash and then
something like Amazon Linux's `ebsnvme-id` can pry it out.

g.