FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3

Tue Oct 17 02:31:30 UTC 2017

On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote:
> Hi Ken,
> 
> I've attached the output of dmesg.  Here's the SCSI CDB for a sample drive, da3.
> 
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 00 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00
> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00
> 
> 
> My understanding is that FreeBSD 11.1 contains the mpr(4) driver?  I've tried this w/ 11.1, w/ the same results.
> 

Yes, the mpr(4) driver is in all recent FreeBSD releases.

In looking at the dmesg, this is telling:

(da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00
(da0:mpr0:0:8:0): CAM status: SCSI Status Error
(da0:mpr0:0:8:0): SCSI status: Check Condition
(da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, format in progress)
(da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete
(da0:mpr0:0:8:0): Error 16, Unretryable error          

If the drives are in the process of formatting, I guess it may make sense
for them to reject read commands.  Otherwise, it makes no sense for a hard
drive to reject reads.

Are you able to check the status of the format?  You should be able to send
a test unit ready and figure out how far along the format is:

camcontrol tur da0 -v

And so on for each of the drives.

Ken

> ________________________________
> From: Kenneth D. Merry [ken at FreeBSD.ORG]
> Sent: Monday, October 16, 2017 7:42 AM
> To: Shiva Bhanujan
> Cc: freebsd-scsi at freebsd.org
> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3
> 
> On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote:
> > Hello,
> >
> > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the following:
> >
> >
> >
> > (da0:mpr0:0:0:0): CAM status: SCSI Status Error
> >
> > (da0:mpr0:0:0:0): SCSI status: Check Condition
> >
> > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code)
> >
> 
> The error message above is missing the SCSI CDB. We need that in order to
> figure out what command the drive is complaining about.
> 
> The error message means that FreeBSD is sending a SCSI command that the
> drive doesn't support. That can be benign, or it can cause a problem.
> 
> So, what error does gpart give you when you have this problem?
> 
> > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also has the same result.
> >
> 
> 
> > Please note, that these errors don't show up on a Fusion-MPT SAS-2 controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either.
> >
> >
> >
> >
> > Is there a different version of the mpr driver I should be using? I haven't checked the differences between the mpr driver in 10.2 vs 10.3 and 11.0. I do see that there are others who have experienced these issues. Can somebody please provide me some pointers
> > as to why this is occurring? Or if there are some driver changes that I might be able to incorporate?
> >
> 
> In general, the latest mpr(4) driver is the best one. The driver itself
> generally doesn't send SCSI commands (there are a few exceptions), but
> rather passes them through from the upper layers of CAM.
> 
> > Please note, that I have gone through the mail titled "scsi error at SEAGATE ST1200MM0088 TT31" and have started sg_format on all the SEAGATE disks. Having said that, I still need to figure out what would happen, if the disks were written to using FreeBSD 10.2, which doesn't seem to have SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated.
> >
> 
> Send the full error messages, and we may be able to figure out what's going
> on.
> 
> Ken
> --
> Kenneth Merry
> ken at FreeBSD.ORG<mailto:ken at FreeBSD.ORG>
> ________________________________

-- 
Kenneth Merry
ken at FreeBSD.ORG