benefit of GEOM labels for ZFS, was Hard drive device names... serial numbers

[Lawrence K. Chen, P.Eng.]

----- Original Message -----
> On Mar 4, 2013, at 5:38 AM, Daniel Kalchev wrote:
> 
> > What you do when the disk is dead, and you don't know which one it
> > is in a rather big rack full of disks?
> > 
> > Perhaps, you offline each and every disk in the system until you
> > eliminate all but one? :)
> 
> One thing that I recently discovered is led(4).
> 
> I have no idea how ubiquitous support for this is, but I see it on
> both a Supermicro and a Tyan board.  In /dev/led, things are
> labelled like so:
> 
> [spork at util2 ~]$ ls -l /dev/led/
> total 0
> crw-------  1 root  wheel    0,  40 Feb  9 00:20 ahcich0.fault
> crw-------  1 root  wheel    0,  39 Feb  9 00:20 ahcich0.locate
> crw-------  1 root  wheel    0,  42 Feb  9 00:20 ahcich1.fault
> crw-------  1 root  wheel    0,  41 Feb  9 00:20 ahcich1.locate
> crw-------  1 root  wheel    0,  44 Feb  9 00:20 ahcich2.fault
> crw-------  1 root  wheel    0,  43 Feb  9 00:20 ahcich2.locate
> crw-------  1 root  wheel    0,  46 Feb  9 00:20 ahcich3.fault
> crw-------  1 root  wheel    0,  45 Feb  9 00:20 ahcich3.locate
> crw-------  1 root  wheel    0,  48 Feb  9 00:20 ahcich4.fault
> crw-------  1 root  wheel    0,  47 Feb  9 00:20 ahcich4.locate
> crw-------  1 root  wheel    0,  50 Feb  9 00:20 ahcich5.fault
> crw-------  1 root  wheel    0,  49 Feb  9 00:20 ahcich5.locate
> 
> If you pair that up with boot messages, you can probably sort out
> which drive is which:
> 
> [spork at util2 ~]$ grep "at ahcich1" /var/run/dmesg.boot
> ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
> 
> And then you can blink the "locate" LED on the sas/sata backplane:
> 
> root at util2:/home/spork # echo f > /dev/led/ahcich0.locate
> 
> And disable the blinking after you're done:
> 
> root at util2:/home/spork # echo 0 > /dev/led/ahcich0.locate
> 
> I'm sure this is all very hardware dependent, but if you have
> supported hardware, it's an easy way to find what's where.
> 
> Charles
> 
> 

Hmmm, interesting....

> ls -l /dev/led
total 0
crw-------  1 root  wheel    0,  59 Feb 27 23:28 ahcich0.fault
crw-------  1 root  wheel    0,  58 Feb 27 23:28 ahcich0.locate
crw-------  1 root  wheel    0,  61 Feb 27 23:28 ahcich1.fault
crw-------  1 root  wheel    0,  60 Feb 27 23:28 ahcich1.locate
crw-------  1 root  wheel    0,  63 Feb 27 23:28 ahcich2.fault
crw-------  1 root  wheel    0,  62 Feb 27 23:28 ahcich2.locate
crw-------  1 root  wheel    0,  65 Feb 27 23:28 ahcich3.fault
crw-------  1 root  wheel    0,  64 Feb 27 23:28 ahcich3.locate
crw-------  1 root  wheel    0,  67 Feb 27 23:28 ahcich4.fault
crw-------  1 root  wheel    0,  66 Feb 27 23:28 ahcich4.locate
crw-------  1 root  wheel    0,  69 Feb 27 23:28 ahcich5.fault
crw-------  1 root  wheel    0,  68 Feb 27 23:28 ahcich5.locate
crw-------  1 root  wheel    0,  48 Feb 27 23:28 siisch0
crw-------  1 root  wheel    0,  49 Feb 27 23:28 siisch1
crw-------  1 root  wheel    0,  54 Feb 27 23:28 siisch2
crw-------  1 root  wheel    0,  55 Feb 27 23:28 siisch3
crw-------  1 root  wheel    0,  56 Feb 27 23:28 siisch4
crw-------  1 root  wheel    0,  57 Feb 27 23:28 siisch5

ahcich is the mobo controller, wonder where the LEDs are...don't recall seeing any when I've been poking around inside.

But, the 10 external drives are on the siisch controllers.  Actually 5 on siisch0 and 5 on siisch1.

> dmesg | grep 'siisch0'
siisch0: <SIIS channel> at channel 0 on siis0
pmp0 at siisch0 bus 0 scbus4 target 15 lun 0
ada4 at siisch0 bus 0 scbus4 target 0 lun 0
ada5 at siisch0 bus 0 scbus4 target 1 lun 0
ada6 at siisch0 bus 0 scbus4 target 2 lun 0
ada7 at siisch0 bus 0 scbus4 target 3 lun 0
ada8 at siisch0 bus 0 scbus4 target 4 lun 0

The enclosure (SANS Digital) on siisch0 has 2 LEDs at each drive position and a LED on the front for each.  The enclosure on siisch1 was cheaper, and lacks the LEDs, but uses the same carriers, etc.  I suppose if it really mattered, I could move the SANS Digital enclosure from my old fileserver to here, making both enclosures on old server the cheaper Rosewill ones....  Though I haven't figured out how to get persistent drive identifiers or such on ubuntu....and, since I'm just doing RAID1 across, and separate VGs....when a drive fails in a RAID set...either there's an LED or its the corresponding one in the other enclosure....

siisch[01] is a Sil3132 and siisch[2345] is a Sil3124

Though I'm thinking of switch one or both for asm1061 based controllers.  Had put the idea on hold at first, but now that my video card is no longer supported by the latest nvidia-driver...I'm considering whether I want to risk getting a newer card (it was the 6th card I tried before I had something mostly working...)  Only some of my drives are SATA III though.  But all 3 SSDs are....and think the ones I'm using for ZIL/L2ARC might benefit.  Perhaps I need 3 asm1061's....

L