[Bug 248906] LSI SAS2008 (mps) gets stuck in a reset loop when writing on AMD Epyc 3000

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248906

            Bug ID: 248906
           Summary: LSI SAS2008 (mps) gets stuck in a reset loop when
                    writing on AMD Epyc 3000
           Product: Base System
           Version: Unspecified
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: paxswill at paxswill.com

Overview:
I'm trying to use an LSI SAS2008 based PCIe card with an AMD Epyc 3151 system.
Once I try to write anything to a drive connected to the card, the mps driver
appears to get stuck in a reset loop, repeating messages like this:

mps0: IOC Fault 0x40002622, Resetting
mps0: Reinitializing controller,
mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd
mps0: IOCCapabilities:
1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
mps0: mps_reinit finished sc 0xfffffe00014a9000 post 4 free 3
mps0: SAS Address for SATA device = 2a04546ea96c8bac
mps0: SAS Address from SATA device = 2a04546ea96c8bac
mps0: SAS Address for SATA device = d9413b15bbcdcc78
mps0: SAS Address from SATA device = d9413b15bbcdcc78

There's then a pause for a few seconds, and these messages are printed again
(none of the values change).

Reproduction Steps:
1. Set up hardware with a SuperMicro M11SDV-4C-LN4F and an LSI SAS2008 HBA PCIe
card that's been reflashed to the IT firmware. Connect a SATA disk to the HBA.
2. Boot FreeBSD (off of install media, another disk, etc).
3. Once booted, check dmesg to see the name of the SATA disk (ex: da0)
4. Run `dd if=/dev/zero of=/dev/da0`

Expected:
Zeros are successfully written to the disk.

Actual:
mps driver gets stuck in a reset loop.

Comments:
* I've tested two different cards (one reflashed by me, another bought off of
eBay pre-flashed), and they both exhibit this issue.
* Ubuntu is able to use both cards.
* I've tested both an SSD and HDD, with no difference.
* This machine is specifically running FreeNAS 11.3-U4.1 (FreeBSD 11.3p11
equivalent). I encountered the same issue with FreeBSD 12.1-RELEASE as well.
* I haven't had a chance to try them in another Intel system yet, but will
update this issue once I have.
* Reads work fine (tested with `dd if=/dev/da0 of=/temp/read_test`). The data
is as expected.
* smartctl is:
    * Able to read SMART values off of drives.
    * Run a background test runs successfully.
    * Running a foreground test fails. After waiting 1 minute, smartctl exits.
Checking the SMART test log shows that the test was "Interrupted (host reset)"
without completing, and these messages are logged by the system:

        (pass1:mps0:0:5:0): ATA COMMAND PASS THROUGH(16). CDB: 85 06 0c 00 d4
00 00 00 81 00 4f 00 c2 00 b0 00 length 0 SMID 700 Aborting command
0xfffffe00015246c0
mps0: Sending reset from mpssas_send_abort for target ID 5
mps0: Unfreezing devq for target ID

The

-- 
You are receiving this mail because:
You are the assignee for the bug.