From nobody Sun Mar 30 23:45:36 2025 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZQrVT1pzXz5sGkM; Sun, 30 Mar 2025 23:45:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZQrVT0X6Jz40lJ; Sun, 30 Mar 2025 23:45:37 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1743378337; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=qBF73yvCwiecHmIWJZrN8Do3R4bg8ktEmDPGUnK2bkE=; b=PKfQwA9//Mt+7iAK96LmJgJE8wDeEIM2E7FCLTnx671AZSasLfdw++q7jiNRGfedtw3Q6H 2izMV2LEk1AZZLUi7L9bK8c3hRpyVJY6W/I05DleJ5MZytmdNbjcYKV3KgPtnHZ8d9xfT6 JUDFrE49fMyy7XcG4t6rpYTpgDojKFQ4gy+H8mhjmQlxuHBv/7k9/B1NaXD1DBqWknfJnA qS8KtHmqzXKBbLWhSe73HNzPt+Ugez70vzQvLGqVHXHXLEoORGclkN9DxWpOkuAuDkOdp4 8gP+nNzQIFQsQIcWe0RSe+7/8X5u9mwCsSEv2/kSts3ZlBXtELn2ctJsG0uiuw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1743378337; a=rsa-sha256; cv=none; b=O0eKz9IgU4H+s6dv4QN72sdRilv0ZDtN6BILBOpPiEdB2Q4hOemJdFLl0XHRv5GqwHn8Hw W2HtpeAdA4lY34ZBPHMGBuGImNslsLm73JOOmbMPDvbPW+OMnsPNRmckDtsmNrXNtvzTQY 6ksiHTKlgALfth0Owe/7QFTYALy7tdoQLEUolBOhWHjQ3AbHD0jR+ItJ+c461TRnKQqQV+ uCocO+QmKUQaz/soHj9VRtHcuFPW3IJvCxtFRfRXEiGUf31TIu+anUT+w0MrXvFw1nYPyH Pbg5G+/iGmdDnPpgxM/VsotZ/PzinLE20l1BrO7UdwStEmicBy93f7bVOQB7BA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1743378337; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=qBF73yvCwiecHmIWJZrN8Do3R4bg8ktEmDPGUnK2bkE=; b=S+YmcDKU+GL1NDYYhnD5km4GYmnk/Ri1jMh0R9V21EDmi9LSBaeWmFJmn2576maY1XuaNs vXb/FpECB/gyJNI+OxsZfeGfhkav5ji1pQblM5cDreFw6CI5ItmVAVwz1I8pIHtcy2q+ke uqEqiG+EabavilYyfiAPqiefTUQ4fEZOd5r9p40FvIKgtEHNnWI7Hn+lK3xv2IqYk5i5Rv yFdwo1uxeSAWaqMh+Yh0yHCQC4sFhH+gKkS4iwYogQk2PCcIzazQE+2S5Tohi56XiVRfka orVu49jWAL88vYIViasyTQUHoZkCesBcyjaACEaZrxs5IzR1f4SgNrRgJiTa3Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4ZQrVS6pBXz4hM; Sun, 30 Mar 2025 23:45:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 52UNjawv032727; Sun, 30 Mar 2025 23:45:36 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 52UNjaC0032724; Sun, 30 Mar 2025 23:45:36 GMT (envelope-from git) Date: Sun, 30 Mar 2025 23:45:36 GMT Message-Id: <202503302345.52UNjaC0032724@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Colin Percival Subject: git: 25df691800f0 - stable/14 - nvme: Fix hotplug on one of the amazon platforms List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: cperciva X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 25df691800f08756228d3ac52ac64d80e9fe998d Auto-Submitted: auto-generated The branch stable/14 has been updated by cperciva: URL: https://cgit.FreeBSD.org/src/commit/?id=25df691800f08756228d3ac52ac64d80e9fe998d commit 25df691800f08756228d3ac52ac64d80e9fe998d Author: Warner Losh AuthorDate: 2025-02-25 16:29:14 +0000 Commit: Colin Percival CommitDate: 2025-03-30 23:44:57 +0000 nvme: Fix hotplug on one of the amazon platforms Amazon EC2 m7i cloud instances use PCI hotplug rather than ACPI hotplug. The card is removed and detach is called to remove the drive from the system. The hardware is no longer present at this point, but the bridge doesn't translate the now-missing hardware reads to all ff's leading us to conclude the hardware is there and we need to do a proper shutdown of it. Fix this oversight by asking the bridge if the device is still present as well. We need both tests since some systems one cane remove the card w/o a hotplug event and we want to fail-safe in those cases. Convert gone to a bool while I'm here and update a comment about shutting down the controller and why that's important. Tested by: cperciva Sponsored by: Netflix (cherry picked from commit dc95228d98474aba940e3885164912b419c5579d) --- sys/dev/nvme/nvme_ctrlr.c | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/sys/dev/nvme/nvme_ctrlr.c b/sys/dev/nvme/nvme_ctrlr.c index 5a825c10f584..6f5d6ae74add 100644 --- a/sys/dev/nvme/nvme_ctrlr.c +++ b/sys/dev/nvme/nvme_ctrlr.c @@ -1513,7 +1513,8 @@ nvme_ctrlr_construct(struct nvme_controller *ctrlr, device_t dev) void nvme_ctrlr_destruct(struct nvme_controller *ctrlr, device_t dev) { - int gone, i; + int i; + bool gone; ctrlr->is_dying = true; @@ -1523,10 +1524,16 @@ nvme_ctrlr_destruct(struct nvme_controller *ctrlr, device_t dev) goto noadminq; /* - * Check whether it is a hot unplug or a clean driver detach. - * If device is not there any more, skip any shutdown commands. + * Check whether it is a hot unplug or a clean driver detach. If device + * is not there any more, skip any shutdown commands. Some hotplug + * bridges will return zeros instead of ff's when the device is + * departing, so ask the bridge if the device is gone. Some systems can + * remove the drive w/o the bridge knowing its gone (they don't really + * do hotplug), so failsafe with detecting all ff's (impossible with + * this hardware) as the device being gone. */ - gone = (nvme_mmio_read_4(ctrlr, csts) == NVME_GONE); + gone = bus_child_present(dev) == 0 || + (nvme_mmio_read_4(ctrlr, csts) == NVME_GONE); if (gone) nvme_ctrlr_fail(ctrlr); else @@ -1554,17 +1561,17 @@ nvme_ctrlr_destruct(struct nvme_controller *ctrlr, device_t dev) nvme_admin_qpair_destroy(&ctrlr->adminq); /* - * Notify the controller of a shutdown, even though this is due to - * a driver unload, not a system shutdown (this path is not invoked - * during shutdown). This ensures the controller receives a - * shutdown notification in case the system is shutdown before - * reloading the driver. + * Notify the controller of a shutdown, even though this is due to a + * driver unload, not a system shutdown (this path is not invoked uring + * shutdown). This ensures the controller receives a shutdown + * notification in case the system is shutdown before reloading the + * driver. Some NVMe drives need this to flush their cache to stable + * media and consider it a safe shutdown in SMART stats. */ - if (!gone) + if (!gone) { nvme_ctrlr_shutdown(ctrlr); - - if (!gone) nvme_ctrlr_disable(ctrlr); + } noadminq: if (ctrlr->taskqueue)