From nobody Fri Jul 15 19:51:06 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Ll26V3Y61z4WTZk for ; Fri, 15 Jul 2022 19:51:14 +0000 (UTC) (envelope-from ambrisko@ambrisko.com) Received: from mail2.ambrisko.com (mail2.ambrisko.com [70.91.206.91]) by mx1.freebsd.org (Postfix) with ESMTP id 4Ll26T30GRz3lP1; Fri, 15 Jul 2022 19:51:13 +0000 (UTC) (envelope-from ambrisko@ambrisko.com) IronPort-SDR: uGf1gHeIJw03Qs5wlZGc6SMQGo16Ttmf5VbI44yXjxp9Sr12aX1aqrKYSoRNiyb/p86TQanedy z+4nRnh4dTgN7WXmNQPLs4CaEYAAYj0UU= X-Ambrisko-Me: Yes IronPort-Data: A9a23:4859X63roG6XXOTRS/bD5cxxkn2cJEfYwER7XKvMYLTBsI5bp2BSn zcYCGrTPqveNGvyL9twYYS/9xkPvpCBn4BnHAZq+CA2RRqmiyZl6fd1j6vUF3nPRiH7oc4OA /w2MrEsFuhtJpPhjkzF3obJ/CAUOZ6gFuKU5N7sYkiddCc8IMsToUsLd90R3uaEteOE7zal4 rselSF/1GiNgFaYOkpMg06KRYgGUP7a4Fv0tXRmDRxHUcO3e9D40fsiya+Nw3vQGuG4H8a7Q frO1rew+iXQ+h03C8imlfDwdUhirrz6ZFnUzCMIC+77xEIqSi8ais7XMNIVbE1Nii6KmPh4z d9XtIezTkEiOaikdOE1DkUAQn4jVUFB0PqdSZSliuSXwlfud3b2yOl0SkYsMuUw9fx6BGtJ3 fICJT0HdRzFgPi5qJq+Q/dEiN4uIcPwMMUYoH4I5T/DAPssWpyGSLjQ/9JewB8+nM1DF+3SI c0DZlJSgL7oC/FUEkwaEowzhr3uj3z1aTxDq1XTrq0yi1U/BTdZiNDFWOc5sPTTLSmMtkrH9 G/A4UrjBRQWaI6WxTafqCv+j+rFhyLgW4U6Hbiy7P9xg1rVzWsWUUVEWVy+qPi/q0i/R9MPd hRNq3Z29fA/pB6xU93wfxyku3rY7BQSbMVdTr8h4waXx6uKvwvAXjoYTiRMYcANvdMtQWB4z UeAmt7kXGQ9sLCcRX+H2K2TqDe+ZXocIWMYP3ZWRA4P+dj4o4YbhxfFVNd4E6nzhdrwQGmiz zePpSk4prMSkc9biv3irAyf22qh/8GbQBQ06wPbWnOewjl4PIP1NZa17VX77OpbKNrLRFe2o 3VZydOV6/oDDM/RmXXVEvkNBryg+92MLCbY3Qx0B5Ak+jmgpyyjcIRX7G0sLUtlKJxdKz7vf ELJvwpVopZWNmGrdqxwJYm2Dp1yn6TnEN3kUNHSb8ZPMsUpLV7bpHk2aB7CxX3pnWgtjbo7a MWSfsubBHoHDbhqkWitTOAH3L53ni0zmTHJSZbgw0j12LaSfiTNG6wIKkWDdLp/5aaOugTO8 NEZPMyPkk0NXOr7ayjR0IgSMVFacCBiVMyu85RaJryZPw5rOGA9EPuAk7oudrtsk7lRiuqVr Gq2XVVVyQaniHDKQelQhquPtF87sU5DkE8G IronPort-HdrOrdr: A9a23:N7Idj6l63C0Lkt+xFpDyjn9A9LPpDfIq3DAbv31ZSRFFG/FwWf rOoB0+726StN93YgBHpTngAtjlfZqyz/JICOUqUotKGTOWwVdAT7sSiLcKoQeQeBEWn9Q1vc wMT0E9MqyTMbEQt6bHCWeDferJ8LO8mpyVuQ== Received: from server2.ambrisko.com (HELO internal.ambrisko.com) ([192.168.1.2]) by ironport2.ambrisko.com with ESMTP; 15 Jul 2022 11:42:53 -0700 Received: from ambrisko.com (localhost [127.0.0.1]) by internal.ambrisko.com (8.17.1/8.17.1) with ESMTPS id 26FJp6Bh074103 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 15 Jul 2022 12:51:06 -0700 (PDT) (envelope-from ambrisko@ambrisko.com) X-Authentication-Warning: internal.ambrisko.com: Host localhost [127.0.0.1] claimed to be ambrisko.com Received: (from ambrisko@localhost) by ambrisko.com (8.17.1/8.17.1/Submit) id 26FJp6qu074102; Fri, 15 Jul 2022 12:51:06 -0700 (PDT) (envelope-from ambrisko) Date: Fri, 15 Jul 2022 12:51:06 -0700 From: Doug Ambrisko To: Larry Rosenman Cc: Michael Gmelin , Alexander Motin , Freebsd current Subject: Re: SAS/SATA controllers: 8 port that support 8TB Drives Message-ID: References: <32B35839-84F9-4BDC-B8B5-AF576E21041C@freebsd.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 4Ll26T30GRz3lP1 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of ambrisko@ambrisko.com has no SPF policy when checking 70.91.206.91) smtp.mailfrom=ambrisko@ambrisko.com X-Spamd-Result: default: False [-2.00 / 15.00]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.995]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; R_SPF_NA(0.00)[no SPF record]; MLMMJ_DEST(0.00)[freebsd-current]; ARC_NA(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:7922, ipnet:70.88.0.0/14, country:US]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[ambrisko]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[ambrisko.com]; HAS_XAW(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Fri, Jun 17, 2022 at 05:55:50PM -0500, Larry Rosenman wrote: | On 06/17/2022 5:48 pm, Michael Gmelin wrote: | >> On 18. Jun 2022, at 00:31, Alexander Motin wrote: | >> | >>  | >> | >>> On 17.06.2022 18:24, Alexander Motin wrote: | >>>> On 17.06.2022 18:16, Larry Rosenman wrote: | >>>> On 06/17/2022 5:08 pm, Alexander Motin wrote: | >>>>> On 17.06.2022 11:59, Larry Rosenman wrote: | >>>>>> I'm looking to upgrade the controllers in my TrueNAS box to | >>>>>> something that will | >>>>>> support 8TB drives because apparently my LSI 2108 controllers do | >>>>>> not support 8TB drives. | >>>>>> | >>>>>> What's the communities recommendation? | >>>>>> needs to support SFF connectors for a total of 4 SFF connectors, | >>>>>> as I have 16 slots. | >>>>> | >>>>> We at iX are still using LSI/Broadcom HBAs, just moved from long | >>>>> discontinued mps(4) to newer mpr(4). And I don't believe the | >>>>> problem | >>>>> is directly related to capacity. According to my observations it | >>>>> may | >>>>> be Seagate HDDs of/above certain (8TB) generation. We do not use | >>>>> Seagate HDDs in our products, so about that instability I only | >>>>> heard | >>>>> from forums and TrueNAS community user reports. | >>>> | >>>> This is a mfi(4) set of controllers, and a ST80000Nm0045 8TB (CMR) | >>>> drive. | >>>> | >>>> Is this a bad combo? | >>>> | >>>> mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not | >>>> supported | >>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | >>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | >>>> (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain | >>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | >>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | >>>> (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain | >>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | >>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | >>>> (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain | >>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | >>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | >>>> (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain | >>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 | >>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an error | >>>> (probe0:mfi0:0:0:0): Error 5, Retries exhausted | >>>> | >>>> mfi0 Physical Drives: | >>>> 0 ( 932G) UNCONFIGURED GOOD >>> serial=ZA1AC912> SATA E1:S3 | >>> mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS | >>> due to problems with hot-plug, disk identification, etc. and so have | >>> limited experience with them. But I know some of LSI RAIDs can be | >>> reflashed into equivalent HBAs, so if they share the hardware, I can | >>> speculate that they may share some issues. | >> | >> I've just noticed "932G" instead of "8000G". It is obviously a bigger | >> problem than what we heard for HBAs. It looks like a kind of problems | >> that should not happen to HBAs, since they should not care about disk | >> capacity. | >> | > | > What does `smartctl -a ` report (especially sector sizes)? | > | > -m | > | > | >> -- | >> Alexander Motin | >> | It's not even making a mfid* node (it is a 4Kn disk) FYI, mfi never got support for 4Kn drives and is hard coded to 512. mrsas does have support so that is better to use for 4Kn and newer controllers. My day job isn't in that area anymore. I looked a bit as what it would take to support 4Kn in mfi and that required a bunch of things and structures to figure it out. Doug A.