From nobody Tue Nov 03 04:33:41 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jJbLM0Zl2z6xsDs for ; Tue, 03 Nov 2026 04:33:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jJbLL3jbbz4fY8 for ; Tue, 03 Nov 2026 04:33:46 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793680426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9GzMsMhueDZ5LPlp+YfJJYDX6ybNCdENvOv0+h1q5+g=; b=yoEH8s0xbNVymEvCPnwjpmjDZq80YbcqEegSA67qq4bhA8igQN/Lez6mB85gbVMFCYB6/m ueutbAEYSzYWwxx+f1pz7ez1pV8Gi6MWsgO1Qxy+W7qgCIPymEPHbT9VVtkjPBgK/doMxO 3ZCS/YIi6wmSmKxzrko6iS0bOoSiZwAe5WEbIcQO88/IKAe0adqVJ0OYSHHyyfesMMqQya Ji8KHgX3td2pT6brdpHbNy3ry/VYC+iJWrCQVfoK3U/0mWEiCnVOYaPqxqf1FiABEMIT1x 6KQjWqvkMSJ9+7zZVjLtzWVWN9hWzdN6vbwsXxK8VOvPh/WwoQNWEa3xUSPN9w== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793680426; b=IrrmzDVxkxSNlMuq3l3P0W3aw5OXyH3Zj/kF1qiaJZFaq5w+AwCn/7bTctM1VsLvcppxy5 gbK5nvOnAQf1d7Oghj/VIb/+oMtrCTNKDBwHWW0jTZ96TNiSo30d5csePTZXnwUQN6LUZy cge9lgzRvuqnWUW1iP40JbU+D2SZEF+thz7lkzhhY2Pk/VNShmDnxQW8F5XAgekoUqaOnO 0LJMZuVMFFhTtDX+cJ0XA0zgayOl4LdX53776LCLGLw1KMzcKpSD6WdU8x8ZN9PfKhLcV5 TfkHd1OebfEKbXn7edNhMYl/WTkE4UAw7z8Kuu66WpJCBpb1W9aq4YBzOebDhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793680426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9GzMsMhueDZ5LPlp+YfJJYDX6ybNCdENvOv0+h1q5+g=; b=scq3XSCz54NxkjZnjm+8rWxn4t7C2OuUuNxl3WSm6aZfBMOwp0C3HkiaNoLo4tX/tO/dcU Omt+U4qtbohwASuYl7rk9P6mALQBfiuFciC+OM8Hw7ixwEN26m0o3qEQOJv6xQIkiAvf8J 8VvXVoQYs5M3No+lOzJ83y8DSVkUm9MDHFg24xujKs78uak9z15dmRPJ2eGXQGfLeAXUy9 1nuOh/GokCavQ2LRuW2BeMPb2HZlCVkCydhUndpJlhW4iya1HgRrQDFz7gHf+66LHsxavo g1a2BFzJk9tPtouOhy3IvIVSa6tSg3k19nn1rxoUsaBDpBTZDzf/zkWhxHeI3A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jJbLL35w1z1G4v for ; Tue, 03 Nov 2026 04:33:46 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 20293 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 03 Nov 2026 04:33:41 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Jaeyoon Choi Subject: git: 5603ddb8299d - main - ufshci: detect completions with the completion notification register List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jaeyoon X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 5603ddb8299d0fe3b8e81a5bee5cd1fa24a1806c Auto-Submitted: auto-generated Date: Tue, 03 Nov 2026 04:33:41 +0000 Message-Id: <6ae96425.20293.60f67c6b@gitrepo.freebsd.org> The branch main has been updated by jaeyoon: URL: https://cgit.FreeBSD.org/src/commit/?id=5603ddb8299d0fe3b8e81a5bee5cd1fa24a1806c commit 5603ddb8299d0fe3b8e81a5bee5cd1fa24a1806c Author: Jaeyoon Choi AuthorDate: 2026-11-03 04:11:14 +0000 Commit: Jaeyoon Choi CommitDate: 2026-11-03 04:29:41 +0000 ufshci: detect completions with the completion notification register The single doorbell completion scan took a cleared doorbell bit as a completion. A bit that was never set reads the same, so a slot whose doorbell was not rung yet, or one that a reset wiped, looked complete and was finished with a stale descriptor. UFSHCI 3.0 added UTRLCNR, which the controller sets when it completes a request and the driver already clears afterwards. Use it to find completed slots on a 3.0 or later controller. Older controllers keep the doorbell check, and the task management list has no such register. Reviewed by: imp Sponsored by: Samsung Electronics Differential Revision: https://reviews.freebsd.org/D61111 --- sys/dev/ufshci/ufshci_private.h | 6 +++--- sys/dev/ufshci/ufshci_req_queue.c | 4 ++-- sys/dev/ufshci/ufshci_req_sdb.c | 22 +++++++++++++++------- 3 files changed, 20 insertions(+), 12 deletions(-) diff --git a/sys/dev/ufshci/ufshci_private.h b/sys/dev/ufshci/ufshci_private.h index 66cbe4a81bb5..d3a02f45f271 100644 --- a/sys/dev/ufshci/ufshci_private.h +++ b/sys/dev/ufshci/ufshci_private.h @@ -133,7 +133,7 @@ struct ufshci_qops { struct ufshci_tracker **tr); void (*ring_doorbell)(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); - bool (*is_doorbell_cleared)(struct ufshci_controller *ctrlr, + bool (*is_completed)(struct ufshci_controller *ctrlr, uint8_t slot); void (*clear_cpl_ntf)(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); @@ -532,9 +532,9 @@ void ufshci_req_sdb_utmr_ring_doorbell(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); void ufshci_req_sdb_utr_ring_doorbell(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); -bool ufshci_req_sdb_utmr_is_doorbell_cleared(struct ufshci_controller *ctrlr, +bool ufshci_req_sdb_utmr_is_completed(struct ufshci_controller *ctrlr, uint8_t slot); -bool ufshci_req_sdb_utr_is_doorbell_cleared(struct ufshci_controller *ctrlr, +bool ufshci_req_sdb_utr_is_completed(struct ufshci_controller *ctrlr, uint8_t slot); void ufshci_req_sdb_utmr_clear_cpl_ntf(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); diff --git a/sys/dev/ufshci/ufshci_req_queue.c b/sys/dev/ufshci/ufshci_req_queue.c index f1b145f9791b..add399e98ed2 100644 --- a/sys/dev/ufshci/ufshci_req_queue.c +++ b/sys/dev/ufshci/ufshci_req_queue.c @@ -28,7 +28,7 @@ static const struct ufshci_qops sdb_utmr_qops = { .reserve_slot = ufshci_req_sdb_reserve_slot, .reserve_admin_slot = ufshci_req_sdb_reserve_slot, .ring_doorbell = ufshci_req_sdb_utmr_ring_doorbell, - .is_doorbell_cleared = ufshci_req_sdb_utmr_is_doorbell_cleared, + .is_completed = ufshci_req_sdb_utmr_is_completed, .clear_cpl_ntf = ufshci_req_sdb_utmr_clear_cpl_ntf, .process_cpl = ufshci_req_sdb_process_cpl, .get_inflight_io = ufshci_req_sdb_get_inflight_io, @@ -43,7 +43,7 @@ static const struct ufshci_qops sdb_utr_qops = { .reserve_slot = ufshci_req_sdb_reserve_slot, .reserve_admin_slot = ufshci_req_sdb_reserve_slot, .ring_doorbell = ufshci_req_sdb_utr_ring_doorbell, - .is_doorbell_cleared = ufshci_req_sdb_utr_is_doorbell_cleared, + .is_completed = ufshci_req_sdb_utr_is_completed, .clear_cpl_ntf = ufshci_req_sdb_utr_clear_cpl_ntf, .process_cpl = ufshci_req_sdb_process_cpl, .get_inflight_io = ufshci_req_sdb_get_inflight_io, diff --git a/sys/dev/ufshci/ufshci_req_sdb.c b/sys/dev/ufshci/ufshci_req_sdb.c index b737a7081fd9..2404354bfade 100644 --- a/sys/dev/ufshci/ufshci_req_sdb.c +++ b/sys/dev/ufshci/ufshci_req_sdb.c @@ -551,21 +551,31 @@ ufshci_req_sdb_utr_ring_doorbell(struct ufshci_controller *ctrlr, } bool -ufshci_req_sdb_utmr_is_doorbell_cleared(struct ufshci_controller *ctrlr, +ufshci_req_sdb_utmr_is_completed(struct ufshci_controller *ctrlr, uint8_t slot) { uint32_t utmrldbr; + /* The task management list has no completion notification. */ utmrldbr = ufshci_mmio_read_4(ctrlr, utmrldbr); return (!(utmrldbr & (1 << slot))); } bool -ufshci_req_sdb_utr_is_doorbell_cleared(struct ufshci_controller *ctrlr, +ufshci_req_sdb_utr_is_completed(struct ufshci_controller *ctrlr, uint8_t slot) { - uint32_t utrldbr; + uint32_t utrlcnr, utrldbr; + /* + * Since UFSHCI 3.0 the controller sets a notification bit when it + * completes a request. Before that, a cleared doorbell bit was the + * only sign, and a slot that was never rung looks the same. + */ + if (ctrlr->major_version >= 3) { + utrlcnr = ufshci_mmio_read_4(ctrlr, utrlcnr); + return ((utrlcnr & (1 << slot)) != 0); + } utrldbr = ufshci_mmio_read_4(ctrlr, utrldbr); return (!(utrldbr & (1 << slot))); } @@ -592,15 +602,13 @@ ufshci_req_sdb_process_cpl(struct ufshci_req_queue *req_queue) KASSERT(tr, ("there is no tracker assigned to the slot")); /* - * When the response is delivered from the device, the doorbell - * is cleared. Check it under qlock so that a slot whose + * Check the completion under qlock so that a slot whose * doorbell write is still in flight in the submit path is not * mistaken for a completed one. */ mtx_lock(&hwq->qlock); completed = tr->slot_state == UFSHCI_SLOT_STATE_SCHEDULED && - req_queue->qops.is_doorbell_cleared(req_queue->ctrlr, - slot); + req_queue->qops.is_completed(req_queue->ctrlr, slot); mtx_unlock(&hwq->qlock); if (completed) { ufshci_req_queue_complete_tracker(tr);