From nobody Tue Nov 03 04:33:41 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jJbLF34nQz6xsLc for ; Tue, 03 Nov 2026 04:33:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jJbLF2T6Dz4fZw for ; Tue, 03 Nov 2026 04:33:41 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793680421; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9GzMsMhueDZ5LPlp+YfJJYDX6ybNCdENvOv0+h1q5+g=; b=otkGcqaviD/qcn9KO40l7g21A/P11Prc9twIaghI+z6Aeb/rbIwCVbp7H0NoNttB8WMGYc CrNyuFUWs5UytwDEyqpZk75I8cz1LPsYsdnwFCJldO03s2b7d8oQEEBmhiRUmP1g50W7yp zQ41af3+Vt4FDkNtyIq9ITsNDPDVd0KYOBjQSbo3SNNDtwCnHAmK1n93YhjrzWtJGTX2pE X0Wbx48hn5l9l4H9L+YanKXjQ/lEEEo1X2cTS5WkzIbRNfpZPZ0rzW/jGlhIKiDw3XhGTL JhxCrlGRxVou+8ZlzGyu+Ig+uI+q/xa3VXaVPOh57P0gBCppwP8mKmly+b1huw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793680421; b=IoOuTrfBnCLfRzc3eacLvOyKuXsOHFS7Q3IDnUlV8Xqyl08oQCHg0ImEHvvo6b6yZtgAVF kitbNp0l2+vAW3PZXIKC83gykOtijgMxc5n+Nl2rBpT5DncrGl2hxkZXwmAtKPGxRpQqZs EjdwDQFVwgceTRjgvPhbtLe7xsBwtGs7YlcGL5Z3Aq9i/tJOluXN1J6rQnb+ebHAQTJUNy +kx6DfgEDKk92XSS+f98WO6rEWI4cx7ySr5lXSVcUtyZZvHjiDcT5WiGkIR1qWWDAOlBGb B/1Wy1dzOlScOv9dEskJ7v5kfLas3CwO61H/GZwMNuwU4nOI6KxmVnIYHX/j0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793680421; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9GzMsMhueDZ5LPlp+YfJJYDX6ybNCdENvOv0+h1q5+g=; b=UcEOx4aP5TTViODhIL+rGbhA4YtdaErgbZiaVHfmbX1oY3MezbfTAk8Yzl/hDMQ9TVlgrZ 9jSKSYDuJIdPBq1v9ynpmjbtMibiKzZ9Am2MfLISsnZz2tRJdcBylPtnMTYvpUzO8i5O/6 dKB+zda9TvJTQqK5Ky5PV9zminrf/IjeQXlQ7ypgR4uZHGLykUoMATJDAQzLNNJXK4MKFk BJfembZwy65601m3pLFxQcaECMXWucMMkIkffzf/HFIwILbT+fb2XftmSmvrEOawuyYbj2 bLCgat7OEkgHPBR92FejnKPsxH6vWlcDN4xD+wh9r8nCM0evFYoPUCKZqjmdhQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jJbLF24S7z1GXW for ; Tue, 03 Nov 2026 04:33:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 20293 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 03 Nov 2026 04:33:41 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Jaeyoon Choi Subject: git: 5603ddb8299d - main - ufshci: detect completions with the completion notification register List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: Commit messages for the main branch of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jaeyoon X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 5603ddb8299d0fe3b8e81a5bee5cd1fa24a1806c Auto-Submitted: auto-generated Date: Tue, 03 Nov 2026 04:33:41 +0000 Message-Id: <6ae96425.20293.60f67c6b@gitrepo.freebsd.org> The branch main has been updated by jaeyoon: URL: https://cgit.FreeBSD.org/src/commit/?id=5603ddb8299d0fe3b8e81a5bee5cd1fa24a1806c commit 5603ddb8299d0fe3b8e81a5bee5cd1fa24a1806c Author: Jaeyoon Choi AuthorDate: 2026-11-03 04:11:14 +0000 Commit: Jaeyoon Choi CommitDate: 2026-11-03 04:29:41 +0000 ufshci: detect completions with the completion notification register The single doorbell completion scan took a cleared doorbell bit as a completion. A bit that was never set reads the same, so a slot whose doorbell was not rung yet, or one that a reset wiped, looked complete and was finished with a stale descriptor. UFSHCI 3.0 added UTRLCNR, which the controller sets when it completes a request and the driver already clears afterwards. Use it to find completed slots on a 3.0 or later controller. Older controllers keep the doorbell check, and the task management list has no such register. Reviewed by: imp Sponsored by: Samsung Electronics Differential Revision: https://reviews.freebsd.org/D61111 --- sys/dev/ufshci/ufshci_private.h | 6 +++--- sys/dev/ufshci/ufshci_req_queue.c | 4 ++-- sys/dev/ufshci/ufshci_req_sdb.c | 22 +++++++++++++++------- 3 files changed, 20 insertions(+), 12 deletions(-) diff --git a/sys/dev/ufshci/ufshci_private.h b/sys/dev/ufshci/ufshci_private.h index 66cbe4a81bb5..d3a02f45f271 100644 --- a/sys/dev/ufshci/ufshci_private.h +++ b/sys/dev/ufshci/ufshci_private.h @@ -133,7 +133,7 @@ struct ufshci_qops { struct ufshci_tracker **tr); void (*ring_doorbell)(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); - bool (*is_doorbell_cleared)(struct ufshci_controller *ctrlr, + bool (*is_completed)(struct ufshci_controller *ctrlr, uint8_t slot); void (*clear_cpl_ntf)(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); @@ -532,9 +532,9 @@ void ufshci_req_sdb_utmr_ring_doorbell(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); void ufshci_req_sdb_utr_ring_doorbell(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); -bool ufshci_req_sdb_utmr_is_doorbell_cleared(struct ufshci_controller *ctrlr, +bool ufshci_req_sdb_utmr_is_completed(struct ufshci_controller *ctrlr, uint8_t slot); -bool ufshci_req_sdb_utr_is_doorbell_cleared(struct ufshci_controller *ctrlr, +bool ufshci_req_sdb_utr_is_completed(struct ufshci_controller *ctrlr, uint8_t slot); void ufshci_req_sdb_utmr_clear_cpl_ntf(struct ufshci_controller *ctrlr, struct ufshci_tracker *tr); diff --git a/sys/dev/ufshci/ufshci_req_queue.c b/sys/dev/ufshci/ufshci_req_queue.c index f1b145f9791b..add399e98ed2 100644 --- a/sys/dev/ufshci/ufshci_req_queue.c +++ b/sys/dev/ufshci/ufshci_req_queue.c @@ -28,7 +28,7 @@ static const struct ufshci_qops sdb_utmr_qops = { .reserve_slot = ufshci_req_sdb_reserve_slot, .reserve_admin_slot = ufshci_req_sdb_reserve_slot, .ring_doorbell = ufshci_req_sdb_utmr_ring_doorbell, - .is_doorbell_cleared = ufshci_req_sdb_utmr_is_doorbell_cleared, + .is_completed = ufshci_req_sdb_utmr_is_completed, .clear_cpl_ntf = ufshci_req_sdb_utmr_clear_cpl_ntf, .process_cpl = ufshci_req_sdb_process_cpl, .get_inflight_io = ufshci_req_sdb_get_inflight_io, @@ -43,7 +43,7 @@ static const struct ufshci_qops sdb_utr_qops = { .reserve_slot = ufshci_req_sdb_reserve_slot, .reserve_admin_slot = ufshci_req_sdb_reserve_slot, .ring_doorbell = ufshci_req_sdb_utr_ring_doorbell, - .is_doorbell_cleared = ufshci_req_sdb_utr_is_doorbell_cleared, + .is_completed = ufshci_req_sdb_utr_is_completed, .clear_cpl_ntf = ufshci_req_sdb_utr_clear_cpl_ntf, .process_cpl = ufshci_req_sdb_process_cpl, .get_inflight_io = ufshci_req_sdb_get_inflight_io, diff --git a/sys/dev/ufshci/ufshci_req_sdb.c b/sys/dev/ufshci/ufshci_req_sdb.c index b737a7081fd9..2404354bfade 100644 --- a/sys/dev/ufshci/ufshci_req_sdb.c +++ b/sys/dev/ufshci/ufshci_req_sdb.c @@ -551,21 +551,31 @@ ufshci_req_sdb_utr_ring_doorbell(struct ufshci_controller *ctrlr, } bool -ufshci_req_sdb_utmr_is_doorbell_cleared(struct ufshci_controller *ctrlr, +ufshci_req_sdb_utmr_is_completed(struct ufshci_controller *ctrlr, uint8_t slot) { uint32_t utmrldbr; + /* The task management list has no completion notification. */ utmrldbr = ufshci_mmio_read_4(ctrlr, utmrldbr); return (!(utmrldbr & (1 << slot))); } bool -ufshci_req_sdb_utr_is_doorbell_cleared(struct ufshci_controller *ctrlr, +ufshci_req_sdb_utr_is_completed(struct ufshci_controller *ctrlr, uint8_t slot) { - uint32_t utrldbr; + uint32_t utrlcnr, utrldbr; + /* + * Since UFSHCI 3.0 the controller sets a notification bit when it + * completes a request. Before that, a cleared doorbell bit was the + * only sign, and a slot that was never rung looks the same. + */ + if (ctrlr->major_version >= 3) { + utrlcnr = ufshci_mmio_read_4(ctrlr, utrlcnr); + return ((utrlcnr & (1 << slot)) != 0); + } utrldbr = ufshci_mmio_read_4(ctrlr, utrldbr); return (!(utrldbr & (1 << slot))); } @@ -592,15 +602,13 @@ ufshci_req_sdb_process_cpl(struct ufshci_req_queue *req_queue) KASSERT(tr, ("there is no tracker assigned to the slot")); /* - * When the response is delivered from the device, the doorbell - * is cleared. Check it under qlock so that a slot whose + * Check the completion under qlock so that a slot whose * doorbell write is still in flight in the submit path is not * mistaken for a completed one. */ mtx_lock(&hwq->qlock); completed = tr->slot_state == UFSHCI_SLOT_STATE_SCHEDULED && - req_queue->qops.is_doorbell_cleared(req_queue->ctrlr, - slot); + req_queue->qops.is_completed(req_queue->ctrlr, slot); mtx_unlock(&hwq->qlock); if (completed) { ufshci_req_queue_complete_tracker(tr);