sesutil not seeing disk in first bay

John jwd at freebsd.org
Fri Mar 10 14:36:11 UTC 2017


Sorry for the slow response, system was being moved.

----- Alan Somers's Original Message -----
> Hard to say whether the problem is in sesutil or in your expander's
> firmware.  Could you please install sysutils/sg3_utils and run "sg_ses
> -p2 /dev/ses0" and "sg_ses -p10 /dev/ses0" ?

Requested outputs:

sg_ses -p2 /dev/ses0 - https://people.freebsd.org/~jwd/ses0.sg_ses-p2.txt

sg_ses -p10 /dev/ses0 - https://people.freebsd.org/~jwd/ses0.sg_ses-p10.txt

For completeness:

camcontrol smpphylist ses0 - https://people.freebsd.org/~jwd/ses0.camcontrol-smpphylist.txt

sesutil -u /dev/ses0 map - https://people.freebsd.org/~jwd/ses0.sesutil-map.txt

Using a combination of smpphylist to get the phy & disk seems
to always work in conjunction with sg_ses -p 10 to map the phy/disk
to a slot number by matching the phy.

sesutil seems like it's off by 1.

Thanks,
John

> -Alan
> 
> On Fri, Feb 24, 2017 at 7:00 AM, John <jwd at freebsd.org> wrote:
> > ----- Alexander Motin's Original Message -----
> >> Hi.
> >>
> >> I am not sure I completely understand from your description how exactly
> >> detected disks match their enclosure slots, but make sure to take a look
> >> on this change:
> >> https://svnweb.freebsd.org/base?view=revision&revision=308425
> >
> >    Yes - this patch is on my system.
> >
> >    To phrase it differently - the shelf contains 25 drives. Disks da0
> > through da23 show up via sesutil:
> >
> > # sesutil map -u /dev/ses0 | grep da
> >                 Device Names: da0,pass0
> >                 Device Names: da1,pass1
> >                 Device Names: da2,pass2
> >                 Device Names: da3,pass3
> >                 Device Names: da4,pass4
> >                 Device Names: da5,pass5
> >                 Device Names: da6,pass6
> >                 Device Names: da7,pass7
> >                 Device Names: da8,pass8
> >                 Device Names: da9,pass9
> >                 Device Names: da10,pass10
> >                 Device Names: da11,pass11
> >                 Device Names: da12,pass12
> >                 Device Names: da13,pass13
> >                 Device Names: da14,pass14
> >                 Device Names: da15,pass15
> >                 Device Names: da16,pass16
> >                 Device Names: da17,pass17
> >                 Device Names: da18,pass18
> >                 Device Names: da19,pass19
> >                 Device Names: da20,pass20
> >                 Device Names: da21,pass21
> >                 Device Names: da22,pass22
> >                 Device Names: da23,pass23
> >
> > Disk da24 does not show up but is present in the first bay. da0
> > is in bay 2, etc.
> >
> > I hope this helps.
> >
> > Thanks,
> > John
> >
> >> On 24.02.2017 12:27, John wrote:
> >> > Hi Folks -
> >> >
> >> >    Running fresh fbsd 11.0-STABLE #1 r314197: Fri Feb 24 05:05:34 EST 2017.
> >> >
> >> >    I'm wondering if anyone using sesutil has seen the following issue
> >> > with an HP D2700 shelf (25 x 600GB SAS drives):
> >> >
> >> > sesutil map -u /dev/ses0
> >> > ses0:
> >> >         Enclosure Name: HP D2700 SAS AJ941A 0149
> >> >         Enclosure ID: 5001438023432c00
> >> >         Element 0, Type: Array Device Slot
> >> >                 Status: Unsupported (0x00 0x80 0x00 0x00)
> >> >         Element 1, Type: Array Device Slot
> >> >                 Status: OK (0x01 0x80 0x00 0x00)
> >> >         Element 2, Type: Array Device Slot
> >> >                 Status: OK (0x01 0x80 0x00 0x00)
> >> >                 Device Names: da0,pass0
> >> >         Element 3, Type: Array Device Slot
> >> >                 Status: OK (0x01 0x80 0x00 0x00)
> >> >                 Device Names: da1,pass1
> >> >         Element 4, Type: Array Device Slot
> >> >                 Status: OK (0x01 0x80 0x00 0x00)
> >> >                 Device Names: da2,pass2
> >> >         ...
> >> >         ... full list below
> >> >         ...
> >> >     Element 25, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da23,pass23
> >> >
> >> >
> >> > Note Element 1 is an does not contain any device names.
> >> >
> >> > Sesutil is getting an ENODEV back from the ioctl requesting the names.
> >> >
> >> > In scsi_enc.c the error seems be coming from here:
> >> >
> >> >                 cam_periph_lock(periph);
> >> >                 error = (*enc->enc_vec.get_elm_devnames)(enc, &elmdn);
> >> >                 cam_periph_unlock(periph);
> >> >
> >> > which eventually calls cam_periph_list() which "looks" ok to me.
> >> >
> >> > I'm wondering if there is some sort of off by 1 error somewhere in
> >> > the code with respect to how this shelf reports devices and are
> >> > assigned since da0 is not allocated to the first bay, but instead bay 2.
> >> >
> >> > Device da24 (the 25th device) is in the first bay. If I run a dd or
> >> > iozone to da24 the drive works correctly and the activity light
> >> > comes on as expected.
> >> >
> >> > This process works correctly on a set of supermicro shelves.
> >> >
> >> > Any thoughts or pointers?
> >> >
> >> > Thanks,
> >> > John
> >> >
> >> > dmesg for da0/da1
> >> >
> >> > (da0:mps0:0:8:0): UNMAPPED
> >> > da0 at mps0 bus 0 scbus0 target 8 lun 0
> >> > da0: <HP EG0600FBDSR HPD6> Fixed Direct Access SPC-3 SCSI device
> >> > da0: Serial Number EA01PC91LLBB1239
> >> > da0: 600.000MB/s transfers
> >> > da0: Command Queueing enabled
> >> > da0: 572325MB (1172123568 512 byte sectors)
> >> > (da1:mps0:0:9:0): UNMAPPED
> >> > da1 at mps0 bus 0 scbus0 target 9 lun 0
> >> > da1: <HP EG0600FBDSR HPD6> Fixed Direct Access SPC-3 SCSI device
> >> > da1: Serial Number EA01PC91LPG91239
> >> > da1: 600.000MB/s transfers
> >> > da1: Command Queueing enabled
> >> > da1: 572325MB (1172123568 512 byte sectors)
> >> >
> >> >
> >> >
> >> > # sesutil map -u /dev/ses0
> >> > ses0:
> >> >     Enclosure Name: HP D2700 SAS AJ941A 0149
> >> >     Enclosure ID: 5001438023432c00
> >> >     Element 0, Type: Array Device Slot
> >> >             Status: Unsupported (0x00 0x80 0x00 0x00)
> >> >     Element 1, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >     Element 2, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da0,pass0
> >> >     Element 3, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da1,pass1
> >> >     Element 4, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da2,pass2
> >> >     Element 5, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da3,pass3
> >> >     Element 6, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da4,pass4
> >> >     Element 7, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da5,pass5
> >> >     Element 8, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da6,pass6
> >> >     Element 9, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da7,pass7
> >> >     Element 10, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da8,pass8
> >> >     Element 11, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da9,pass9
> >> >     Element 12, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da10,pass10
> >> >     Element 13, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da11,pass11
> >> >     Element 14, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da12,pass12
> >> >     Element 15, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da13,pass13
> >> >     Element 16, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da14,pass14
> >> >     Element 17, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da15,pass15
> >> >     Element 18, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da16,pass16
> >> >     Element 19, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da17,pass17
> >> >     Element 20, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da18,pass18
> >> >     Element 21, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da19,pass19
> >> >     Element 22, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da20,pass20
> >> >     Element 23, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da21,pass21
> >> >     Element 24, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da22,pass22
> >> >     Element 25, Type: Array Device Slot
> >> >             Status: OK (0x01 0x80 0x00 0x00)
> >> >             Device Names: da23,pass23
> >> >     Element 26, Type: SAS Connector
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 27, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 28, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 29, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 30, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 31, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 32, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 33, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 34, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 35, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 36, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 37, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 38, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 39, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 40, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 41, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 42, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 43, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 44, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 45, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 46, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 47, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 48, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 49, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 50, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 51, Type: SAS Connector
> >> >             Status: OK (0x01 0x20 0x00 0x00)
> >> >     Element 52, Type: <Type 0xff>
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 53, Type: <Type 0xff>
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >             Description:     5C7236P2L5
> >> >     Element 54, Type: Power Supply
> >> >             Status: Unsupported (0x00 0x00 0x00 0x80)
> >> >     Element 55, Type: Power Supply
> >> >             Status: OK (0x01 0x00 0x00 0x80)
> >> >             Description:   20Delta            5ANLD0C4D3A0XN
> >> >     Element 56, Type: Power Supply
> >> >             Status: OK (0x01 0x00 0x00 0x80)
> >> >             Description:   20Delta            5ANLD0C4D3A0XS
> >> >     Element 57, Type: Cooling
> >> >             Status: OK (0x01 0x00 0x00 0x80)
> >> >             Description:       B PAVCA0B9V3B0GH
> >> >             Extra status:
> >> >             - Speed: 0 rpm
> >> >     Element 58, Type: Cooling
> >> >             Status: Unsupported (0x00 0x02 0x8f 0x02)
> >> >             Extra status:
> >> >             - Speed: 6550 rpm
> >> >     Element 59, Type: Cooling
> >> >             Status: Unsupported (0x00 0x02 0x6b 0x02)
> >> >             Extra status:
> >> >             - Speed: 6190 rpm
> >> >     Element 60, Type: Cooling
> >> >             Status: OK (0x01 0x00 0x00 0x80)
> >> >             Description:       B PAVCA0B9V3B0DL
> >> >             Extra status:
> >> >             - Speed: 0 rpm
> >> >     Element 61, Type: Cooling
> >> >             Status: Unsupported (0x00 0x02 0xa3 0x02)
> >> >             Extra status:
> >> >             - Speed: 6750 rpm
> >> >     Element 62, Type: Cooling
> >> >             Status: Unsupported (0x00 0x02 0x6b 0x02)
> >> >             Extra status:
> >> >             - Speed: 6190 rpm
> >> >     Element 63, Type: Enclosure Services Controller Electronics
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 64, Type: Enclosure Services Controller Electronics
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >             Description: 014901B0PAZAV0BTM3D3AX  0026
> >> >     Element 65, Type: SAS Connector
> >> >             Status: Unsupported (0x00 0x02 0x00 0x00)
> >> >     Element 66, Type: SAS Connector
> >> >             Status: OK (0x01 0x02 0x00 0x00)
> >> >     Element 67, Type: SAS Connector
> >> >             Status: OK (0x01 0x02 0x01 0x00)
> >> >     Element 68, Type: SAS Connector
> >> >             Status: OK (0x01 0x02 0x02 0x00)
> >> >     Element 69, Type: SAS Connector
> >> >             Status: OK (0x01 0x02 0x03 0x00)
> >> >     Element 70, Type: SAS Connector
> >> >             Status: Unsupported (0x00 0x02 0x00 0x00)
> >> >     Element 71, Type: SAS Connector
> >> >             Status: OK (0x01 0x02 0x00 0x00)
> >> >     Element 72, Type: SAS Connector
> >> >             Status: OK (0x01 0x02 0x01 0x00)
> >> >     Element 73, Type: SAS Connector
> >> >             Status: OK (0x01 0x02 0x02 0x00)
> >> >     Element 74, Type: SAS Connector
> >> >             Status: OK (0x01 0x02 0x03 0x00)
> >> >     Element 75, Type: SAS Expander
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 76, Type: SAS Expander
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >             Description: 900080050002
> >> >     Element 77, Type: Display
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >             Description: 0.107SEGDISP C:03
> >> >     Element 78, Type: Display
> >> >             Status: Unsupported (0x00 0x02 0x30 0x00)
> >> >     Element 79, Type: Display
> >> >             Status: Unsupported (0x00 0x02 0x31 0x00)
> >> >     Element 80, Type: Temperature Sensors
> >> >             Status: Unsupported (0x00 0x00 0x43 0x00)
> >> >             Extra status:
> >> >             - Temperature: 47 C
> >> >     Element 81, Type: Temperature Sensors
> >> >             Status: OK (0x01 0x00 0x43 0x00)
> >> >             Extra status:
> >> >             - Temperature: 47 C
> >> >     Element 82, Type: Enclosure Services Controller Electronics
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 83, Type: Enclosure Services Controller Electronics
> >> >             Status: Unknown (0x06 0x00 0x00 0x00)
> >> >             Description: 0149                    0026
> >> >     Element 84, Type: SAS Connector
> >> >             Status: Unsupported (0x00 0x02 0x00 0x00)
> >> >     Element 85, Type: SAS Connector
> >> >             Status: Unknown (0x06 0x02 0x00 0x00)
> >> >     Element 86, Type: SAS Connector
> >> >             Status: Unknown (0x06 0x02 0x01 0x00)
> >> >     Element 87, Type: SAS Connector
> >> >             Status: Unknown (0x06 0x02 0x02 0x00)
> >> >     Element 88, Type: SAS Connector
> >> >             Status: Unknown (0x06 0x02 0x03 0x00)
> >> >     Element 89, Type: SAS Connector
> >> >             Status: Unsupported (0x00 0x02 0x00 0x00)
> >> >     Element 90, Type: SAS Connector
> >> >             Status: Unknown (0x06 0x02 0x00 0x00)
> >> >     Element 91, Type: SAS Connector
> >> >             Status: Unknown (0x06 0x02 0x01 0x00)
> >> >     Element 92, Type: SAS Connector
> >> >             Status: Unknown (0x06 0x02 0x02 0x00)
> >> >     Element 93, Type: SAS Connector
> >> >             Status: Unknown (0x06 0x02 0x03 0x00)
> >> >     Element 94, Type: SAS Expander
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 95, Type: SAS Expander
> >> >             Status: Unknown (0x06 0x00 0x00 0x00)
> >> >             Description:
> >> >     Element 96, Type: Display
> >> >             Status: Unknown (0x06 0x00 0x00 0x00)
> >> >             Description:
> >> >     Element 97, Type: Display
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 98, Type: Display
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 99, Type: Temperature Sensors
> >> >             Status: Unsupported (0x00 0x00 0x43 0x00)
> >> >             Extra status:
> >> >             - Temperature: 47 C
> >> >     Element 100, Type: Temperature Sensors
> >> >             Status: OK (0x01 0x00 0x43 0x00)
> >> >             Extra status:
> >> >             - Temperature: 47 C
> >> >     Element 101, Type: Enclosure Services Controller Electronics
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 102, Type: Enclosure Services Controller Electronics
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >             Description: 002803D0PARHV0DTM3B0FF
> >> >     Element 103, Type: Enclosure Services Controller Electronics
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 104, Type: Enclosure Services Controller Electronics
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >             Description: 1.02
> >> >     Element 105, Type: Enclosure Services Controller Electronics
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >             Description: 1.02
> >> >     Element 106, Type: Temperature Sensors
> >> >             Status: Unsupported (0x00 0x00 0x30 0x00)
> >> >             Extra status:
> >> >             - Temperature: 28 C
> >> >     Element 107, Type: Temperature Sensors
> >> >             Status: OK (0x01 0x00 0x31 0x00)
> >> >             Extra status:
> >> >             - Temperature: 29 C
> >> >     Element 108, Type: Temperature Sensors
> >> >             Status: OK (0x01 0x00 0x30 0x00)
> >> >             Extra status:
> >> >             - Temperature: 28 C
> >> >     Element 109, Type: Power Supply
> >> >             Status: Unsupported (0x00 0x00 0x00 0x00)
> >> >     Element 110, Type: Power Supply
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >     Element 111, Type: Power Supply
> >> >             Status: OK (0x01 0x00 0x00 0x00)
> >> >
> >>
> >> --
> >> Alexander Motin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 585 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-scsi/attachments/20170310/88a6c8e7/attachment.sig>


More information about the freebsd-scsi mailing list