From nobody Fri Jan 21 02:27:01 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4E8951968FD8; Fri, 21 Jan 2022 02:27:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jg3DR0LzJz4p84; Fri, 21 Jan 2022 02:27:03 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642732024; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=J22Q2WIaBXib9DRaXlBeK4DjBdcpoQSvmpDIFJtPzAM=; b=Cp5F2OCIKMiDQ5CLOIRjDQj0bVZXSV2y1rbTVcxXEx9fpOlLyaeynNLvtmQNSJtSy4otHd l8lU03L92HhrkJNbEECU609Pb0PaqpXDMCK8KEuX8vcoIVKRbNtSG/vNjd35H12x1FaEYF FX2CIJ9kzCuifSbAeibuoUCkzBwJDkG9wdvWQp8/+NJAbub3l2Bxy6ncWcgVpYS+3buotj OUMFLfMQB7WRpvjmaroBKQG8GRnBR7fDfROThbQPvsi9Y1m6fGLjnwOEih5arXcyccWgC1 JY6WXe/PBt5IRhnFhpY5FSXQyn1SNxL+5gsv2L8Pl4DoFK1eO8dY35VcYTsGsQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CBF3113CB0; Fri, 21 Jan 2022 02:27:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20L2R1QZ001432; Fri, 21 Jan 2022 02:27:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20L2R1uk001431; Fri, 21 Jan 2022 02:27:01 GMT (envelope-from git) Date: Fri, 21 Jan 2022 02:27:01 GMT Message-Id: <202201210227.20L2R1uk001431@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 50b3d57e7161 - stable/13 - nvme: Only reset once on attach. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 50b3d57e7161cb111892b377e301d56bdb5b627a Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642732024; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=J22Q2WIaBXib9DRaXlBeK4DjBdcpoQSvmpDIFJtPzAM=; b=kbkSrYeWCj58bjPXU4GCpcqk81YDym+bjCuTDMgZDt1rgFCiBdWJg5+ABeWjEJV1BE0q8F PFIcfLyWDa31LgB8vRJfS0vAJubLNvnnzRNEa17kDtLay1EozHO33u4/G9G3ov4tD88RuY ajNttyNUWEES7/dq2FA90KhyiHzrSwZtsJhDzrRDHg/I31uDgzcJcBIgMEezYQrGA/X9eX VfOv1+WWp8NFLsZjAA8gRay4zjS52sCQdlKDaDc67IALlATJMrpthnQUdF8ubhButOmX/W mPv8kHkMCsQhzNkfYD1b0CoETFKdQZrSN3IcGxzamMEvRDHhtI4zx3T5zkO1nw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1642732024; a=rsa-sha256; cv=none; b=oe7AR+KahP/Vn8gQUhYO29JK2aK9OVC7SDjm6BG9aqlJZz+REeMKqBCfv/axljIbe+Mk6s biOBFgr4nfGzL+G/PPvvEKGCcgkpXxBDF2PZ4nrz0oQHs9rK1WdiAypGKh+G/r3aOkOxrB SUjbaNC5wmDpo9QMcPE1DlYCypWRASLzsMaqzYMGRYkt9h7HQhGTCD8xpox6Ms1ZBa3f0T YVo+9xmRaD7d7oH5hbJQdnx9bgTil8g0iKrIKFuqdFUYNVKW/3ALRd9SIsr7PTg33cfCmA grImdHmPgVtReFW+ndF/aYybgeZeqwTo+7N0EP+dfWKTB5+tkPXuFFVn75VeHA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=50b3d57e7161cb111892b377e301d56bdb5b627a commit 50b3d57e7161cb111892b377e301d56bdb5b627a Author: Warner Losh AuthorDate: 2021-10-01 17:09:34 +0000 Commit: Alexander Motin CommitDate: 2022-01-21 02:07:31 +0000 nvme: Only reset once on attach. The FreeBSD nvme driver has reset the nvme controller twice on attach to address a theoretical issue assuring the hardware is in a known state. However, exierence has shown the second reset is unnecessary and increases the time to boot. Eliminate the second reset. Should there be a situation when you need a second reset (for buggy or at least somewhat out of the mainstream hardware), the hardware option NVME_2X_RESET will restore the old behavior. Document this in nvme(4). If there's any trouble at all with this, I'll add a sysctl tunable to control it. Sponsored by: Netflix Reviewed by: cperciva, mav Differential Revision: https://reviews.freebsd.org/D32241 (cherry picked from commit 4b3da659bf62b0f5306b5acee9add41b84361498) --- share/man/man4/nvme.4 | 8 ++++++++ sys/conf/options | 1 + sys/dev/nvme/nvme_ctrlr.c | 27 +++++++++++++++++---------- 3 files changed, 26 insertions(+), 10 deletions(-) diff --git a/share/man/man4/nvme.4 b/share/man/man4/nvme.4 index db31b6f26b3e..eb9f9b222f51 100644 --- a/share/man/man4/nvme.4 +++ b/share/man/man4/nvme.4 @@ -172,6 +172,14 @@ value in hw.nvme.verbose_cmd_dump=1 .Ed .Pp +Prior versions of the driver reset the card twice on boot. +This proved to be unnecessary and inefficient, so the driver now resets drive +controller only once. +The old behavior may be restored in the kernel config file with +.Bd -literal -offset indent +.Cd options NVME_2X_RESET +.Ed +.Pp .Sh SYSCTL VARIABLES The following controller-level sysctls are currently implemented: .Bl -tag -width indent diff --git a/sys/conf/options b/sys/conf/options index 6827c236a5d6..78377086315b 100644 --- a/sys/conf/options +++ b/sys/conf/options @@ -1009,6 +1009,7 @@ EKCD opt_ekcd.h # NVME options NVME_USE_NVD opt_nvme.h +NVME_2X_RESET opt_nvme.h # amdsbwd options AMDSBWD_DEBUG opt_amdsbwd.h diff --git a/sys/dev/nvme/nvme_ctrlr.c b/sys/dev/nvme/nvme_ctrlr.c index d0be0da39902..ca89e99d2934 100644 --- a/sys/dev/nvme/nvme_ctrlr.c +++ b/sys/dev/nvme/nvme_ctrlr.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); #include "opt_cam.h" +#include "opt_nvme.h" #include #include @@ -1135,12 +1136,6 @@ nvme_ctrlr_start_config_hook(void *arg) TSENTER(); - /* - * Reset controller twice to ensure we do a transition from cc.en==1 to - * cc.en==0. This is because we don't really know what status the - * controller was left in when boot handed off to OS. Linux doesn't do - * this, however. If we adopt that policy, see also nvme_ctrlr_resume(). - */ if (nvme_ctrlr_hw_reset(ctrlr) != 0) { fail: nvme_ctrlr_fail(ctrlr); @@ -1148,8 +1143,17 @@ fail: return; } +#ifdef NVME_2X_RESET + /* + * Reset controller twice to ensure we do a transition from cc.en==1 to + * cc.en==0. This is because we don't really know what status the + * controller was left in when boot handed off to OS. Linux doesn't do + * this, however, and when the controller is in state cc.en == 0, no + * I/O can happen. + */ if (nvme_ctrlr_hw_reset(ctrlr) != 0) goto fail; +#endif nvme_qpair_reset(&ctrlr->adminq); nvme_admin_qpair_enable(&ctrlr->adminq); @@ -1672,14 +1676,17 @@ nvme_ctrlr_resume(struct nvme_controller *ctrlr) if (ctrlr->is_failed) return (0); - /* - * Have to reset the hardware twice, just like we do on attach. See - * nmve_attach() for why. - */ if (nvme_ctrlr_hw_reset(ctrlr) != 0) goto fail; +#ifdef NVME_2X_RESET + /* + * Prior to FreeBSD 13.1, FreeBSD's nvme driver reset the hardware twice + * to get it into a known good state. However, the hardware's state is + * good and we don't need to do this for proper functioning. + */ if (nvme_ctrlr_hw_reset(ctrlr) != 0) goto fail; +#endif /* * Now that we've reset the hardware, we can restart the controller. Any