Spare drive becoming unavailable zpool

[Jim Ohlstein]

Hello,

I've recently set up a new system that will be used primarily as a file 
server. Install was 10.2-RELEASE with ZFS root. I upgraded to 11-CURRENT 
(r292334) to take advantage of the latest bhyve functionality.

The machine has 12 SAS drives. I set up an 11 drive raidz-3 and set the 
last one as a spare. Twice I've seen the following after a reboot:

# zpool status
   pool: zroot
  state: ONLINE
   scan: scrub repaired 0 in 0h18m with 0 errors on Tue Dec 15 23:42:25 2015
config:

	NAME                    STATE     READ WRITE CKSUM
	zroot                   ONLINE       0     0     0
	  raidz3-0              ONLINE       0     0     0
	    gpt/disk0           ONLINE       0     0     0
	    gpt/disk1           ONLINE       0     0     0
	    gpt/disk2           ONLINE       0     0     0
	    gpt/disk3           ONLINE       0     0     0
	    gpt/disk4           ONLINE       0     0     0
	    gpt/disk5           ONLINE       0     0     0
	    gpt/disk6           ONLINE       0     0     0
	    gpt/disk7           ONLINE       0     0     0
	    gpt/disk8           ONLINE       0     0     0
	    gpt/disk9           ONLINE       0     0     0
	    gpt/disk10          ONLINE       0     0     0
	spares
	  13907025268419731947  UNAVAIL   was /dev/gpt/disk11

The spare has been changed to a long numeric string and is unavailable. 
I can remove it from the pool and re-add it as /dev/gpt/disk11 and 
things seem "normal" Once again it is available as a spare:

# zpool status
   pool: zroot
  state: ONLINE
   scan: scrub repaired 0 in 0h18m with 0 errors on Tue Dec 15 23:42:25 2015
config:

	NAME            STATE     READ WRITE CKSUM
	zroot           ONLINE       0     0     0
	  raidz3-0      ONLINE       0     0     0
	    gpt/disk0   ONLINE       0     0     0
	    gpt/disk1   ONLINE       0     0     0
	    gpt/disk2   ONLINE       0     0     0
	    gpt/disk3   ONLINE       0     0     0
	    gpt/disk4   ONLINE       0     0     0
	    gpt/disk5   ONLINE       0     0     0
	    gpt/disk6   ONLINE       0     0     0
	    gpt/disk7   ONLINE       0     0     0
	    gpt/disk8   ONLINE       0     0     0
	    gpt/disk9   ONLINE       0     0     0
	    gpt/disk10  ONLINE       0     0     0
	spares
	  gpt/disk11    AVAIL

Nothing stands out in smartctl. I've never seen this behavior before. 
Before I ask the host to replace hardware I was wondering if anyone has 
seen this or has any ideas.


-- 
Jim Ohlstein


"Never argue with a fool, onlookers may not be able to tell the 
difference." - Mark Twain