Avago LSI SAS 3008 & Intel SSD Timeouts

[list-news]

Sure Steve:

# cat /boot/loader.conf | grep trim
vfs.zfs.trim.enabled=0

# sysctl vfs.zfs.trim.enabled
vfs.zfs.trim.enabled: 0

# uptime
3:14PM  up 11 mins, 3 users, load averages: 6.58, 11.31, 7.07

# tail -f /var/log/messages:
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): WRITE(10). CDB: 2a 00 2b 
d8 86 50 00 00 b0 00 length 90112 SMID 1016 command timeout cm 
0xfffffe0001375580 ccb 0xfffff8039895f800 target 16, handle(0x0010)
Jun  7 15:13:50 s18 kernel: mpr0: At enclosure level 0, slot 8, 
connector name (    )
Jun  7 15:13:50 s18 kernel: mpr0: timedout cm 0xfffffe0001375580 
allocated tm 0xfffffe0001322150
Jun  7 15:13:50 s18 kernel: (noperiph:mpr0:0:4294967295:0): SMID 1 
Aborting command 0xfffffe0001375580
Jun  7 15:13:50 s18 kernel: mpr0: Sending reset from mprsas_send_abort 
for target ID 16
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): SYNCHRONIZE CACHE(10). 
CDB: 35 00 00 00 00 00 00 00 00 00 length 0 SMID 786 command timeout cm 
0xfffffe00013627a0 ccb 0xfffff8039851e800 target 16, handle(0x0010)
Jun  7 15:13:50 s18 kernel: mpr0: At enclosure level 0, slot 8, 
connector name (    )
Jun  7 15:13:50 s18 kernel: mpr0: queued timedout cm 0xfffffe00013627a0 
for processing by tm 0xfffffe0001322150
Jun  7 15:13:50 s18 kernel: mpr0: EventReply    :
Jun  7 15:13:50 s18 kernel: EventDataLength: 2
Jun  7 15:13:50 s18 kernel: AckRequired: 0
Jun  7 15:13:50 s18 kernel: Event: SasDiscovery (0x16)
Jun  7 15:13:50 s18 kernel: EventContext: 0x0
Jun  7 15:13:50 s18 kernel: Flags: 1<InProgress>
Jun  7 15:13:50 s18 kernel: ReasonCode: Discovery Started
Jun  7 15:13:50 s18 kernel: PhysicalPort: 0
Jun  7 15:13:50 s18 kernel: DiscoveryStatus: 0
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): READ(10). CDB: 28 00 0b 
43 a8 00 00 00 10 00 length 8192 SMID 624 completed cm 
0xfffffe0001355300 ccb 0xfffff803984d4800 during recovery ioc 804b scsi 
0 state c xfer 0
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): READ(10). CDB: 28 00 0b 
43 a8 00 00 00 10 00 length 8192 SMID 624 terminated ioc 804b scsi 0 
state c xfer 0
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): READ(10). CDB: 28 00 0b 
43 a7 f0 00 00 10 00 length 8192 SMID 633 completed cm 
0xfffffe0001355ed0 ccb 0xfffff803987f0000 during recovery ioc 804b scsi 
0 state c xfer 0
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): READ(10). CDB: 28 00 0b 
43 a7 f0 00 00 10 00 length 8192 SMID 633 terminated ioc 804b scsi 0 
state c xfer 0
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): READ(10). CDB: 28 00 0a 
25 3f f0 00 00 08 00 length 4096 SMID 133 completed cm 
0xfffffe000132ce90 ccb 0xfffff803985fc000 during recovery ioc 804b scsi 
0 state c xfer 0
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): READ(10). CDB: 28 00 0a 
25 3f f0 00 00 08 00 length 4096 SMID 133 terminated ioc 804b scsi 0 
state c xfer 0
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): WRITE(10). CDB: 2a 00 2b 
d8 86 50 00 00 b0 00 length 90112 SMID 1016 completed timedout cm 
0xfffffe0001375580 ccb 0xfffff8039895f800 during recovery ioc 8048 scsi 
0 state c    (da6:mpr0:0:16:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 
00 00 00 00 00 00 length 0 SMID 786 completed timedout cm 
0xfffffe(da6:mpr0:0:16:0): WRITE(10). CDB: 2a 00 2b d8 86 50 00 00 b0 00
Jun  7 15:13:50 s18 kernel: 00013627a0 ccb 0xfffff8039851e800 during 
recovery ioc 804b scsi 0 (da6:mpr0:0:16:0): CAM status: Command timeout
Jun  7 15:13:50 s18 kernel: (da6:mpr0:0:16:0): SYNCHRONIZE CACHE(10). 
CDB: 35 00 00 00 00 00 00 00 00 00 length 0 SMID 786 terminated ioc 804b 
scsi 0 sta(da6:te c xfer 0
Jun  7 15:13:50 s18 kernel: mpr0:0: (xpt0:mpr0:0:16:0): SMID 1 abort 
TaskMID 1016 status 0x0 code 0x0 count 5
Jun  7 15:13:50 s18 kernel: 16:    (xpt0:mpr0:0:16:0): SMID 1 finished 
recovery after aborting TaskMID 1016
Jun  7 15:13:50 s18 kernel: 0): mpr0: Retrying command
Jun  7 15:13:50 s18 kernel: Unfreezing devq for target ID 16
Jun  7 15:13:50 s18 kernel: mpr0: EventReply    :
Jun  7 15:13:50 s18 kernel: EventDataLength: 4
Jun  7 15:13:50 s18 kernel: AckRequired: 0
Jun  7 15:13:50 s18 kernel: Event: SasTopologyChangeList (0x1c)
Jun  7 15:13:50 s18 kernel: EventContext: 0x0
Jun  7 15:13:50 s18 kernel: EnclosureHandle: 0x2
Jun  7 15:13:50 s18 kernel: ExpanderDevHandle: 0x9
Jun  7 15:13:50 s18 kernel: NumPhys: 31
Jun  7 15:13:50 s18 kernel: NumEntries: 1
Jun  7 15:13:50 s18 kernel: StartPhyNum: 8
Jun  7 15:13:50 s18 kernel: ExpStatus: Responding (0x3)
Jun  7 15:13:50 s18 kernel: PhysicalPort: 0
Jun  7 15:13:50 s18 kernel: PHY[8].AttachedDevHandle: 0x0010
Jun  7 15:13:50 s18 kernel: PHY[8].LinkRate: 12.0Gbps (0xbb)
Jun  7 15:13:50 s18 kernel: PHY[8].PhyStatus: PHYLinkStatusChange
Jun  7 15:13:50 s18 kernel: mpr0: (0)->(mprsas_fw_work) Working on  
Event: [16]
Jun  7 15:13:50 s18 kernel: mpr0: (1)->(mprsas_fw_work) Event Free: [16]
Jun  7 15:13:50 s18 kernel: mpr0: (1)->(mprsas_fw_work) Working on  
Event: [1c]
Jun  7 15:13:50 s18 kernel: mpr0: (2)->(mprsas_fw_work) Event Free: [1c]
Jun  7 15:13:50 s18 kernel: mpr0: EventReply    :
Jun  7 15:13:50 s18 kernel: EventDataLength: 2
Jun  7 15:13:50 s18 kernel: AckRequired: 0
Jun  7 15:13:50 s18 kernel: Event: SasDiscovery (0x16)
Jun  7 15:13:50 s18 kernel: EventContext: 0x0
Jun  7 15:13:50 s18 kernel: Flags: 0
Jun  7 15:13:50 s18 kernel: ReasonCode: Discovery Complete
Jun  7 15:13:50 s18 kernel: PhysicalPort: 0
Jun  7 15:13:50 s18 kernel: DiscoveryStatus: 0
Jun  7 15:13:50 s18 kernel: mpr0: (2)->(mprsas_fw_work) Working on  
Event: [16]
Jun  7 15:13:50 s18 kernel: mpr0: (3)->(mprsas_fw_work) Event Free: [16]
Jun  7 15:13:51 s18 kernel: (da6:mpr0:0:16:0): SYNCHRONIZE CACHE(10). 
CDB: 35 00 00 00 00 00 00 00 00 00
Jun  7 15:13:51 s18 kernel: (da6:mpr0:0:16:0): CAM status: SCSI Status Error
Jun  7 15:13:51 s18 kernel: (da6:mpr0:0:16:0): SCSI status: Check Condition
Jun  7 15:13:51 s18 kernel: (da6:mpr0:0:16:0): SCSI sense: UNIT 
ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred)
Jun  7 15:13:51 s18 kernel: (da6:mpr0:0:16:0): Retrying command (per 
sense data)

-Kyle

On 6/7/16 2:53 PM, Steven Hartland wrote:
> CDB: 85 is a TRIM command IIRC, I know you tried it before using BIO 
> delete but assuming your running ZFS can you set the following in 
> loader.conf and see how you get on.
> vfs.zfs.trim.enabled=0
>
>     Regards
>     Steve