ZFS on Hardware RAID

Wed Jan 23 15:26:05 UTC 2019

On 1/23/2019 3:38 AM, Ireneusz Pluta wrote:
> W dniu 2019-01-23 o 08:15, Borja Marcos pisze:
>> By the way, some of these “pure raid” cards can be cross flashed to
>> be plain HBAs but it’s a risky procedure. I bricked one or two.
>
> any experience with these ones?
>
> LSI MegaRAID SAS 9280-16i4e
> LSI MegaRAID SAS 9266-4i
> LSI MegaRAID SAS 84016E
>
I have been testing an LSI 9272-8i and it works well with the mrsas
driver. It has JBOD mode.  Using sysutils/storcli (sysutils/megacli
works too) you can interact with the individual disks. I have also used
the 9266, but its jbod doesnt really work (at least with the cards I
tried) and you have to set disks to individual raid0 (MegaCli
-CfgEachDskRaid0 WT NORA Direct CachedBadBBU -Automatic -a0)

storcli /c0 show all (to show disk and enclosure IDs)
storcli /c0 set jbod=on (enable jbod mode for controller)
storcli /c0/e252/s0 set jbod (sets a disk into jbod mode)

smartmontools works with the attached disks as expected.  Performance
seems to be quite good and the controller runs pretty cool.

# zpool status testraid2
  pool: testraid2
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:01:34 with 0 errors on Tue Jan 15
17:34:21 2019
config:

        NAME        STATE     READ WRITE CKSUM
        testraid2   ONLINE       0     0     0
          da1p1     ONLINE       0     0     0
          da3p1     ONLINE       0     0     0
          da4p1     ONLINE       0     0     0
          da5p1     ONLINE       0     0     0

# zpool status testraid
  pool: testraid
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:01:32 with 0 errors on Tue Jan 15
17:34:13 2019
config:

        NAME        STATE     READ WRITE CKSUM
        testraid    ONLINE       0     0     0
          da0       ONLINE       0     0     0
          da2       ONLINE       0     0     0

I created 2 test raid0 arrays and battered the pools with various disk
tests trying to max out the platters/SSDs and all seemed quite stable.

 # pciconf -lvcb mrsas0
mrsas0 at pci0:31:0:0:     class=0x010400 card=0x92721000 chip=0x005b1000
rev=0x05 hdr=0x00
    vendor     = 'LSI Logic / Symbios Logic'
    device     = 'MegaRAID SAS 2208 [Thunderbolt]'
    class      = mass storage
    subclass   = RAID
    bar   [10] = type I/O Port, range 32, base 0xd000, size 256, enabled
    bar   [14] = type Memory, range 64, base 0xf7560000, size 16384, enabled
    bar   [1c] = type Memory, range 64, base 0xf7500000, size 262144,
enabled
    cap 01[50] = powerspec 3  supports D0 D1 D2 D3  current D0
    cap 10[68] = PCI-Express 2 endpoint max data 128(4096) FLR RO NS
                 link x4(x8) speed 5.0(8.0) ASPM disabled(L0s)
    cap 03[d0] = VPD
    cap 05[a8] = MSI supports 1 message, 64 bit
    cap 11[c0] = MSI-X supports 16 messages, enabled
                 Table in map 0x14[0x2000], PBA in map 0x14[0x3000]
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0019[1e0] = PCIe Sec 1 lane errors 0
    ecap 0004[1c0] = Power Budgeting 1
    ecap 0016[190] = DPA 1
    ecap 000e[148] = ARI 1

# storcli /c0 show
Generating detailed summary of the adapter, it may take a while to complete.

CLI Version = 007.0709.0000.0000 Aug 14, 2018
Operating system = FreeBSD 12.0-STABLE
Controller = 0
Status = Success
Description = None

Product Name = LSI MegaRAID SAS 9272-8i
Serial Number = SV41832837
SAS Address =  500605b008d84580
PCI Address = 00:1f:00:00
System Time = 01/23/2019 10:24:57
Mfg. Date = 05/03/14
Controller Time = 01/23/2019 10:25:59
FW Package Build = 23.2.1-0065
BIOS Version = 5.38.00.1_4.12.05.00_0x05210000
FW Version = 3.152.185-3520
Driver Name = mrsas
Driver Version = 07.709.01.00-fbsd

Vendor Id = 0x1000
Device Id = 0x5B
SubVendor Id = 0x1000
SubDevice Id = 0x9272
Host Interface = PCI-E
Device Interface = SAS-6G
Bus Number = 31
Device Number = 0
Function Number = 0
Physical Drives = 6

PD LIST :
=======

---------------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz
Model                     Sp Type
---------------------------------------------------------------------------------------
252:0    14 JBOD  -    2.728 TB SATA HDD N   N  512B WDC
WD30EFRX-68EUZN0      U  -   
252:1    15 JBOD  -    2.728 TB SATA HDD N   N  512B WDC
WD30EFRX-68EUZN0      U  -   
252:2    13 JBOD  -  233.312 GB SATA HDD N   N  512B WDC
WD2502ABYS-02B7A0     U  -   
252:5    11 JBOD  -    931.0 GB SATA HDD N   N  512B WDC
WD1002F9YZ-09H1JL1    U  -   
252:6    12 JBOD  -   465.25 GB SATA SSD N   N  512B WDC
WDS500G2B0A-00SM50    U  -   
252:7    10 JBOD  -   465.25 GB SATA SSD Y   N  512B Samsung SSD 850 EVO
500GB U  -   
---------------------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down/PowerSave|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded|Cpybck-CopyBack|CBShld-Copyback Shielded

    ---Mike

> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>
>

-- 
-------------------
Mike Tancsa, tel +1 519 651 3400 x203
Sentex Communications, mike at sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   