From nobody Fri Jun 17 23:20:09 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5F06F854EC8 for ; Fri, 17 Jun 2022 23:20:13 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.evolve.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LPw4X3Tspz3nRM; Fri, 17 Jun 2022 23:20:12 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: by mail.evolve.de (OpenSMTPD) with ESMTP id 2db81cab; Fri, 17 Jun 2022 23:20:10 +0000 (UTC) Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id 0ca30134 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Fri, 17 Jun 2022 23:20:10 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: SAS/SATA controllers: 8 port that support 8TB Drives From: Michael Gmelin In-Reply-To: Cc: Alexander Motin , Freebsd current Date: Sat, 18 Jun 2022 01:20:09 +0200 Message-Id: <736314DA-354C-465C-8D8C-AF67AAD68F5D@freebsd.org> References: To: Larry Rosenman X-Mailer: iPhone Mail (19E258) X-Rspamd-Queue-Id: 4LPw4X3Tspz3nRM X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 213.239.217.29 is neither permitted nor denied by domain of grembo@freebsd.org) smtp.mailfrom=grembo@freebsd.org X-Spamd-Result: default: False [1.88 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[grembo]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.51)[-0.514]; NEURAL_SPAM_LONG(0.99)[0.990]; MLMMJ_DEST(0.00)[freebsd-current]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N > On 18. Jun 2022, at 00:57, Larry Rosenman wrote: > =EF=BB=BFOn 06/17/2022 5:48 pm, Michael Gmelin wrote: >>> On 18. Jun 2022, at 00:31, Alexander Motin wrote: >>> =EF=BB=BF >>>> On 17.06.2022 18:24, Alexander Motin wrote: >>>>> On 17.06.2022 18:16, Larry Rosenman wrote: >>>>> On 06/17/2022 5:08 pm, Alexander Motin wrote: >>>>>> On 17.06.2022 11:59, Larry Rosenman wrote: >>>>>>> I'm looking to upgrade the controllers in my TrueNAS box to somethin= g that will >>>>>>> support 8TB drives because apparently my LSI 2108 controllers do not= support 8TB drives. >>>>>>> What's the communities recommendation? >>>>>>> needs to support SFF connectors for a total of 4 SFF connectors, as I= have 16 slots. >>>>>> We at iX are still using LSI/Broadcom HBAs, just moved from long >>>>>> discontinued mps(4) to newer mpr(4). And I don't believe the problem= >>>>>> is directly related to capacity. According to my observations it may= >>>>>> be Seagate HDDs of/above certain (8TB) generation. We do not use >>>>>> Seagate HDDs in our products, so about that instability I only heard >>>>>> from forums and TrueNAS community user reports. >>>>> This is a mfi(4) set of controllers, and a ST80000Nm0045 8TB (CMR) dri= ve. >>>>> Is this a bad combo? >>>>> mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not supported= >>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>> (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain >>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>> (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain >>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>> (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain >>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>> (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain >>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error >>>>> (probe0:mfi0:0:0:0): Error 5, Retries exhausted >>>>> mfi0 Physical Drives: >>>>> 0 ( 932G) UNCONFIGURED GOOD SATA E1:S3 >>>> mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS due= to problems with hot-plug, disk identification, etc. and so have limited ex= perience with them. But I know some of LSI RAIDs can be reflashed into equi= valent HBAs, so if they share the hardware, I can speculate that they may sh= are some issues. >>> I've just noticed "932G" instead of "8000G". It is obviously a bigger p= roblem than what we heard for HBAs. It looks like a kind of problems that s= hould not happen to HBAs, since they should not care about disk capacity. >> What does `smartctl -a ` report (especially sector sizes)? >> -m >>> -- >>> Alexander Motin > It's not even making a mfid* node (it is a 4Kn disk) Ok, that=E2=80=99s sad (and explains the wrong size calculation as 4096/512=3D= 8). Is this in HBA mode? (Like Alexander suggested, re-/crossflashing using an I= T firmware might be an option). What controller / firmware image version is i= t? -m > --=20 > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106