Re: SAS/SATA controllers: 8 port that support 8TB Drives
Date: Sat, 18 Jun 2022 08:54:12 UTC
> On 18. Jun 2022, at 01:31, Larry Rosenman <ler@lerctr.org> wrote: > On 06/17/2022 6:20 pm, Michael Gmelin wrote: >>>> On 18. Jun 2022, at 00:57, Larry Rosenman <ler@lerctr.org> wrote: >>> On 06/17/2022 5:48 pm, Michael Gmelin wrote: >>>>> On 18. Jun 2022, at 00:31, Alexander Motin <mav@freebsd.org> wrote: >>>>> >>>>>> On 17.06.2022 18:24, Alexander Motin wrote: >>>>>>> On 17.06.2022 18:16, Larry Rosenman wrote: >>>>>>> On 06/17/2022 5:08 pm, Alexander Motin wrote: >>>>>>>> On 17.06.2022 11:59, Larry Rosenman wrote: >>>>>>>>> I'm looking to upgrade the controllers in my TrueNAS box to something that will >>>>>>>>> support 8TB drives because apparently my LSI 2108 controllers do not support 8TB drives. >>>>>>>>> What's the communities recommendation? >>>>>>>>> needs to support SFF connectors for a total of 4 SFF connectors, as I have 16 slots. >>>>>>>> We at iX are still using LSI/Broadcom HBAs, just moved from long >>>>>>>> discontinued mps(4) to newer mpr(4). And I don't believe the problem >>>>>>>> is directly related to capacity. According to my observations it may >>>>>>>> be Seagate HDDs of/above certain (8TB) generation. We do not use >>>>>>>> Seagate HDDs in our products, so about that instability I only heard >>>>>>>> from forums and TrueNAS community user reports. >>>>>>> This is a mfi(4) set of controllers, and a ST80000Nm0045 8TB (CMR) drive. >>>>>>> Is this a bad combo? >>>>>>> mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported >>>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>>>> (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain >>>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>>>> (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain >>>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>>>> (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain >>>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>>>> (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain >>>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>>>> (probe0:mfi0:0:0:0): Error 5, Retries exhausted >>>>>>> mfi0 Physical Drives: >>>>>>> 0 ( 932G) UNCONFIGURED GOOD <ST8000NM0045-1RL UG07 serial=ZA1AC912> SATA E1:S3 >>>>>> mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due to problems with hot-plug, disk identification, etc. and so have limited experience with them. But I know some of LSI RAIDs can be reflashed into equivalent HBAs, so if they share the hardware, I can speculate that they may share some issues. >>>>> I've just noticed "932G" instead of "8000G". It is obviously a bigger problem than what we heard for HBAs. It looks like a kind of problems that should not happen to HBAs, since they should not care about disk capacity. >>>> What does `smartctl -a <device>` report (especially sector sizes)? >>>> -m >>>>> -- >>>>> Alexander Motin >>> It's not even making a mfid* node (it is a 4Kn disk) >> Ok, that’s sad (and explains the wrong size calculation as 4096/512=8). >> Is this in HBA mode? (Like Alexander suggested, re-/crossflashing >> using an IT firmware might be an option). What controller / firmware >> image version is it? >> -m >>> -- >>> Larry Rosenman http://www.lerctr.org/~ler >>> Phone: +1 214-642-9640 E-Mail: ler@lerctr.org >>> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 > > > mfi0@pci0:8:0:0: class=0x010400 rev=0x05 hdr=0x00 vendor=0x1000 device=0x0079 subvendor=0x1734 subdevice=0x1176 > vendor = 'Broadcom / LSI' > device = 'MegaRAID SAS 2108 [Liberator]' > class = mass storage > subclass = RAID > > mfi1@pci0:3:0:0: class=0x010400 rev=0x05 hdr=0x00 vendor=0x1000 device=0x0079 subvendor=0x1734 subdevice=0x1176 > vendor = 'Broadcom / LSI' > device = 'MegaRAID SAS 2108 [Liberator]' > class = mass storage > subclass = RAID > > mfi0: <LSI MegaSAS Gen2> port 0xd000-0xd0ff mem 0xfbc9c000-0xfbc9ffff,0xfbcc0000 > -0xfbcfffff irq 26 at device 0.0 on pci3 > mfi0: Using MSI > mfi0: Megaraid SAS driver Ver 4.23 > mfi0: FW MaxCmds = 1008, limiting to 128 > mfip0: <SCSI Passthrough Bus> on mfi0 > mfi0: 10014 (708822708s/0x0020/info) - Shutdown command received from host > mfi0: 10015 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 00 > 79/1000/1176/1734) > mfi0: 10016 (boot + 3s/0x0020/info) - Firmware version 2.130.353-2727 > mfi0: 10017 (boot + 6s/0x0020/info) - Package version 12.12.0-0174 > mfi0: 10018 (boot + 6s/0x0020/info) - Board Revision > > mfi1: <LSI MegaSAS Gen2> port 0xc000-0xc0ff mem 0xfba9c000-0xfba9ffff,0xfbac0000 > -0xfbafffff irq 16 at device 0.0 on pci8 > mfi1: Using MSI > mfi1: Megaraid SAS driver Ver 4.23 > mfi1: FW MaxCmds = 1008, limiting to 128 > Subvendor is Fujitsu Siemens - so I guess this is integrated into a system by them. Seems like flashing the 2108 to an IT firmware isn’t an option (based on what I found online). You could check if there are firmware updates available though. How did you configure the drives in the megaraid utility (ctrl-h after boot)? Did you create a RAID-0 for each disk? And what capacity is shown in there? Based on [0], 2108 based controllers don’t support 4kn. IT mode would help (true passthrough), but as written above, I don’t think it’s an option for this model. -m [0] https://bitdeals.tech/blogs/news/4kn-lsi-compatibility-list > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106