FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3

Wed Oct 18 01:59:50 UTC 2017

Please note, that this isn't an issue w/ Toshiba drives.  is this a firmware issue by any chance?

________________________________
From: owner-freebsd-scsi at freebsd.org [owner-freebsd-scsi at freebsd.org] on behalf of Shiva Bhanujan [shiva.bhanujan at quorum.net]
Sent: Tuesday, October 17, 2017 6:08 AM
To: Kenneth D. Merry
Cc: freebsd-scsi at freebsd.org
Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3

Since I started having the SCSI errors, I ended up running sg_format to format the disks. I've found that once the disks are formatted using sg_format, there are no SCSI errors. The errors that show up during the format are towards the end of the dmesg output.

(da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, format in progress)
(da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete

once the format is done, I can successfully format and partition using gpart.

The errors that show up when I try to run gpart for the first time are as follows:

(da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00
(da9:mpr0:0:17:0): CAM status: SCSI Status Error
(da9:mpr0:0:17:0): SCSI status: Check Condition
(da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code)
(da9:mpr0:0:17:0): Error 22, Unretryable error

It seems that it's the read that is failing, and is being tagged as an illegal request. While sg_format will address the issue at hand, this isn't an option for us, because there are appliances that were formatted using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue?

________________________________
From: Kenneth D. Merry [ken at FreeBSD.ORG<mailto:ken at FreeBSD.ORG>]
Sent: Monday, October 16, 2017 7:31 PM
To: Shiva Bhanujan
Cc: freebsd-scsi at freebsd.org<mailto:freebsd-scsi at freebsd.org>
Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3

On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote:
> Hi Ken,
>
> I've attached the output of dmesg. Here's the SCSI CDB for a sample drive, da3.
>
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00
>
>
> My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've tried this w/ 11.1, w/ the same results.
>

Yes, the mpr(4) driver is in all recent FreeBSD releases.

In looking at the dmesg, this is telling:

(da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00
(da0:mpr0:0:8:0): CAM status: SCSI Status Error
(da0:mpr0:0:8:0): SCSI status: Check Condition
(da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, format in progress)
(da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete
(da0:mpr0:0:8:0): Error 16, Unretryable error

If the drives are in the process of formatting, I guess it may make sense
for them to reject read commands. Otherwise, it makes no sense for a hard
drive to reject reads.

Are you able to check the status of the format? You should be able to send
a test unit ready and figure out how far along the format is:

camcontrol tur da0 -v

And so on for each of the drives.

Ken

> ________________________________
> From: Kenneth D. Merry [ken at FreeBSD.ORG<mailto:ken at FreeBSD.ORG><mailto:ken at FreeBSD.ORG>]
> Sent: Monday, October 16, 2017 7:42 AM
> To: Shiva Bhanujan
> Cc: freebsd-scsi at freebsd.org<mailto:freebsd-scsi at freebsd.org><mailto:freebsd-scsi at freebsd.org>
> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3
>
> On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote:
> > Hello,
> >
> > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the following:
> >
> >
> >
> > (da0:mpr0:0:0:0): CAM status: SCSI Status Error
> >
> > (da0:mpr0:0:0:0): SCSI status: Check Condition
> >
> > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code)
> >
>
> The error message above is missing the SCSI CDB. We need that in order to
> figure out what command the drive is complaining about.
>
> The error message means that FreeBSD is sending a SCSI command that the
> drive doesn't support. That can be benign, or it can cause a problem.
>
> So, what error does gpart give you when you have this problem?
>
> > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also has the same result.
> >
>
>
> > Please note, that these errors don't show up on a Fusion-MPT SAS-2 controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either.
> >
> >
> >
> >
> > Is there a different version of the mpr driver I should be using? I haven't checked the differences between the mpr driver in 10.2 vs 10.3 and 11.0. I do see that there are others who have experienced these issues. Can somebody please provide me some pointers
> > as to why this is occurring? Or if there are some driver changes that I might be able to incorporate?
> >
>
> In general, the latest mpr(4) driver is the best one. The driver itself
> generally doesn't send SCSI commands (there are a few exceptions), but
> rather passes them through from the upper layers of CAM.
>
> > Please note, that I have gone through the mail titled "scsi error at SEAGATE ST1200MM0088 TT31" and have started sg_format on all the SEAGATE disks. Having said that, I still need to figure out what would happen, if the disks were written to using FreeBSD 10.2, which doesn't seem to have SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated.
> >
>
> Send the full error messages, and we may be able to figure out what's going
> on.
>
> Ken
> --
> Kenneth Merry
> ken at FreeBSD.ORG<mailto:ken at FreeBSD.ORG><mailto:ken at FreeBSD.ORG><mailto:ken at FreeBSD.ORG>
> ________________________________

--
Kenneth Merry
ken at FreeBSD.ORG<mailto:ken at FreeBSD.ORG><mailto:ken at FreeBSD.ORG>
________________________________
_______________________________________________
freebsd-scsi at freebsd.org<mailto:freebsd-scsi at freebsd.org> mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
To unsubscribe, send any mail to "freebsd-scsi-unsubscribe at freebsd.org<mailto:freebsd-scsi-unsubscribe at freebsd.org>"
________________________________