From nobody Sun Oct 04 17:41:47 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hyVFR5TTFz6cTNN for ; Sun, 04 Oct 2026 17:41:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hyVFR1n2Kz3H32 for ; Sun, 04 Oct 2026 17:41:47 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1791135707; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QTSa2HCsDxA/06dpMMY+QtrZUKZ+A7UnOSijr3Wy0ls=; b=I/ROTB8WySEAA+VD0n0vM97uEed4au69DQ0hGPyFOXg+QdVuqFXFKbPZwOcWOC/G0QXfBy wI5BPF8430P2FfIHsJwzIEeRuF7B9G3OasLiODKUPfuKpmb/oTAuSHXV24zlmOmkZHWUt0 2ECXRWMlqbgfXgxi1F+XwNY6CmlS6LlYUGL2J8XfkWozvMejgVslYp2HP5IIGO8d3o2SW9 Cb2b5WiWNRdNKCs0FiOREW1v23ZyFJIRK3z71JbkacdVTleGdd3kycVkYaif5xAxv4jGTS daf52xIXaU/sORDUtPxc8ZnvhtuWACYRbEzeszw8Z5lOX4WabboSZuFyZ50Z/w== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1791135707; b=MTBOFxLux1mLnIc0o9J3Eqs3UJUMePL3rkOgEFwKUNQYgFSMjnPNnb2yMrCwv91wr8AhsH G8doIjMX+gaPosXz5mwXBGXbmGMvIoP1ryLZs7wq4e4xC49r5Rjlf9UNG9KZJszFK2l8z/ zHh8MeLXs/bt6Jxh3ne+kIxmxJLkidJ+uSjoEAYS6EtHH4Bg78D9LUIBrdnZejMyJhL3aP +72pLefzLbFalGKtD5ZKCZcTr4LhjUZAl36wTpH31DU25I8/Q3Sj6GSLeDPtOTuqZzR4Sf YXYjzYNy8/n5gNQng/6zxGjMy/7sdnp+ZtFnJrp0f+OrfulAu8woONCa6lJz6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1791135707; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QTSa2HCsDxA/06dpMMY+QtrZUKZ+A7UnOSijr3Wy0ls=; b=kLOg5bJ5tkiQE3PQ/UCPd3C0iOn6l/Z8aHq8pW4lQZHp0pL8881kkuk5t1VKoFUM+5e7UJ qwQdaNvjl8mA4Cj9vqo9mrBZ6mM0yNE9eOv/rd8kMApK3+Dppiut6rlsyjPudfwonagfsb 0UvLlQS5mSDKc8GRjSTI7V/CL2/aaPioCMNDLiqdb2GjHhbdNKmhcYfk2pg4+G8qb0oWnC MGbHMvgF/yzAR6+iBjHj1RCqSYRAkD2ipCky7IGSVIiZYnuyEqAByKrvC+TOP1LJyrnAm5 SR8Jkw0dAvoL6YynhL7Qr4P0l5yyrXV9Mjx02wCXU6OtUkRnfjTwwjIxtncTAQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hyVFR0XPHzkTj for ; Sun, 04 Oct 2026 17:41:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 32820 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sun, 04 Oct 2026 17:41:47 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Abdelkader Boudih Subject: git: 0a0490ed551d - main - nvme: do not complete a command when its Abort is not performed List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: seuros X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0a0490ed551d8c89aca16b6b2f1a3b3681b60972 Auto-Submitted: auto-generated Date: Sun, 04 Oct 2026 17:41:47 +0000 Message-Id: <6ac28fdb.32820.4900bba3@gitrepo.freebsd.org> The branch main has been updated by seuros: URL: https://cgit.FreeBSD.org/src/commit/?id=0a0490ed551d8c89aca16b6b2f1a3b3681b60972 commit 0a0490ed551d8c89aca16b6b2f1a3b3681b60972 Author: Abdelkader Boudih AuthorDate: 2026-10-04 17:40:52 +0000 Commit: Abdelkader Boudih CommitDate: 2026-10-04 17:40:52 +0000 nvme: do not complete a command when its Abort is not performed An Abort completion with cdw0 bit 0 set means the controller did not abort the command; the command can still complete later. The driver treated this as aborted anyway: it completed the command itself and freed the CID. When the controller completed the command later, the CID may already belong to a new command, so the new command finished with the old command's status, or the unknown-cid assertion fired on INVARIANTS kernels. The watchdog also kept sending a new Abort for the same command every half second while the first one was still pending. Reviewed by: imp, adrian Differential Revision: https://reviews.freebsd.org/D59634 --- sys/dev/nvme/nvme_private.h | 6 ++++++ sys/dev/nvme/nvme_qpair.c | 38 ++++++++++++++------------------------ 2 files changed, 20 insertions(+), 24 deletions(-) diff --git a/sys/dev/nvme/nvme_private.h b/sys/dev/nvme/nvme_private.h index d801f8285f90..c7095bc34334 100644 --- a/sys/dev/nvme/nvme_private.h +++ b/sys/dev/nvme/nvme_private.h @@ -132,6 +132,11 @@ struct nvme_async_event_request { uint8_t log_page_buffer[NVME_MAX_AER_LOG_SIZE]; }; +/* Progress of a hardware Abort for a timed-out tracker */ +#define NVME_ABORT_NONE 0 /* No abort submitted */ +#define NVME_ABORT_SENT 1 /* Abort submitted, not yet completed */ +#define NVME_ABORT_FAILED 2 /* Abort did not abort the command */ + struct nvme_tracker { TAILQ_ENTRY(nvme_tracker) tailq; struct nvme_request *req; @@ -139,6 +144,7 @@ struct nvme_tracker { sbintime_t deadline; bus_dmamap_t payload_dma_map; uint16_t cid; + uint8_t abort_state; uint64_t *prp; bus_addr_t prp_bus_addr; diff --git a/sys/dev/nvme/nvme_qpair.c b/sys/dev/nvme/nvme_qpair.c index 6411a3909917..c918539ce28c 100644 --- a/sys/dev/nvme/nvme_qpair.c +++ b/sys/dev/nvme/nvme_qpair.c @@ -814,34 +814,18 @@ nvme_abort_complete(void *arg, const struct nvme_completion *status) struct nvme_tracker *tr = arg; /* - * If cdw0 bit 0 == 1, the controller was not able to abort the command - * we requested. We still need to check the active tracker array, to - * cover race where I/O timed out at same time controller was completing - * the I/O. An abort command always is on the admin queue, but affects - * either an admin or an I/O queue, so take the appropriate qpair lock - * for the original command's queue, since we'll need it to avoid races - * with the completion code and to complete the command manually. + * On IANP or Abort failure the command may still complete through + * a deferred abort: the tracker stays live to keep its CID + * reserved, and the timeout handler escalates to a reset. */ mtx_lock(&tr->qpair->lock); - if ((status->cdw0 & 1) == 1 && tr->qpair->act_tr[tr->cid] != NULL) { - /* - * An I/O has timed out, and the controller was unable to abort - * it for some reason. And we've not processed a completion for - * it yet. Construct a fake completion status, and then complete - * the I/O's tracker manually. - */ + if ((nvme_completion_is_error(status) || (status->cdw0 & 1) == 1) && + tr->qpair->act_tr[tr->cid] == tr && + tr->abort_state == NVME_ABORT_SENT) { nvme_printf(tr->qpair->ctrlr, - "abort command failed, aborting command manually\n"); - nvme_qpair_manual_complete_tracker(tr, - NVME_SCT_GENERIC, NVME_SC_ABORTED_BY_REQUEST, 0, ERROR_PRINT_ALL); + "abort command failed, resetting on next timeout pass\n"); + tr->abort_state = NVME_ABORT_FAILED; } - /* - * XXX We don't check status for the possible 'Could not abort because - * excess aborts were submitted to the controller'. We don't prevent - * that, either. Document for the future here, since the standard is - * squishy and only says 'may generate' but implies anything is possible - * including hangs if you exceed the ACL. - */ mtx_unlock(&tr->qpair->lock); } @@ -992,6 +976,7 @@ do_reset: * Timeout expired, abort it or reset controller. */ if (ctrlr->enable_aborts && + tr->abort_state == NVME_ABORT_NONE && tr->req->cb_fn != nvme_abort_complete) { /* * This isn't an abort command, ask for a @@ -999,9 +984,13 @@ do_reset: * queue which will reset the card if it * times out. */ + tr->abort_state = NVME_ABORT_SENT; nvme_ctrlr_cmd_abort(ctrlr, qpair->cid_base + tr->cid, qpair->id, nvme_abort_complete, tr); + } else if (ctrlr->enable_aborts && + tr->abort_state == NVME_ABORT_SENT) { + continue; } else { /* * We have a live command in the card (either @@ -1177,6 +1166,7 @@ _nvme_qpair_submit_request(struct nvme_qpair *qpair, struct nvme_request *req) TAILQ_REMOVE(&qpair->free_tr, tr, tailq); TAILQ_INSERT_TAIL(&qpair->outstanding_tr, tr, tailq); tr->deadline = SBT_MAX; + tr->abort_state = NVME_ABORT_NONE; tr->req = req; req->cmd.cid = qpair->cid_base + tr->cid;