From nobody Fri Jan 21 02:26:53 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 371311968CF5; Fri, 21 Jan 2022 02:26:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jg3DG6ZDYz4nkt; Fri, 21 Jan 2022 02:26:54 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642732015; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=r98DOaqcjyREOlk+Yp42oFVc+WKScUcXrQKU6Pvlpp4=; b=Kz+YLr3PjApsjRDAXSvM0N059tyamia8kTxx4exui8yay5tjI2e7DlmVIvJZhcY+MS2Yew WrnSF4MLor5mLV5V08NH+bd6iLYIFNryL3SGVVd0iNP2LDf2MowMhb/zgeX5Yg2ax/npBW Axizy4HEndH6pwKZokk76XRizX2OQQj/5+01BaoPOk4Vw09FeWfJ9rG6KU2H3Yel2H7KlJ GrkwHycOtoJasdo1rs0wHhv6ACxbxUVL02xXAvjBfPgMIakhOVy/AJRee6K+tBX4N4aUqE XvgvNXdcsfIVw2vkk08OQOwIP1Ng4nCoUjsMSZZ6S2bu67YGKCH0p3CIMbrmGg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E64FC13B52; Fri, 21 Jan 2022 02:26:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20L2QrJS001258; Fri, 21 Jan 2022 02:26:53 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20L2QrAV001257; Fri, 21 Jan 2022 02:26:53 GMT (envelope-from git) Date: Fri, 21 Jan 2022 02:26:53 GMT Message-Id: <202201210226.20L2QrAV001257@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 86990decd77e - stable/13 - nvme: count number of ignored interrupts List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 86990decd77efbf36ba99d951b4f17688a939bdf Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642732015; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=r98DOaqcjyREOlk+Yp42oFVc+WKScUcXrQKU6Pvlpp4=; b=MV5WWzDSWpUI6wcNMOg+pjBljSmU7h+mmll/GnWBxSuBDnka5IoSGjlu74qsADKnncnbgx dmCevT/0AKkq8YPKKiDtusGGcLJOO0XtRfsZPXoUQeSGviCRv+7a3p9D2S1yppYZiMU6Uw xRI4ByoZyxpQZrGyDuqGCUzg4gtncveJUpbzKSsXJCp4qUgX5NBm+/1PP7Vh5CAwwiv60f U3y29BaLMRtYA4kp3KcyYlbbiSLSRq4x6gt1sd+xvRVhhf+G2zJrtQmbn7Z3nkKwmPl9d4 CdNR8W49ajspkB+honMkBCTs0Zb3CWILb/k077Xv2l2pGWi7a+CMkHjzdHCeKw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1642732015; a=rsa-sha256; cv=none; b=GMzS9S8lU/LAs4YIxkBd1m3MrG5+T7Oasc48L/MhMYqpsWgHeBUkwM3D0WYv+iJCAOqbt5 hxBFD3Tq6LmGLMN7kYhXYkFGCq70WFxca8jHmKjdfwK0Lg1y9jc0GvtCPHRfqzS1mdOFp3 JVKqLo4EHMkcnkcz+Pvr9jBJyeykWMTvVLtODSFezdkl5PySI+gO/iTYKEoi0zd9NPPiYF I431xIEr6Szm+9hwpyM2leh8Z7toHdXvyY+15PUkaOHhXimef53w+TYo5H8b8PcxY2ajEX kHpTV1K/BB3pIimegWdpbu8lnGVTf8mGlUpi1p2Ts7tdPGeqNoUbTsLNHrj4iQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=86990decd77efbf36ba99d951b4f17688a939bdf commit 86990decd77efbf36ba99d951b4f17688a939bdf Author: Warner Losh AuthorDate: 2021-09-29 03:13:11 +0000 Commit: Alexander Motin CommitDate: 2022-01-21 02:07:30 +0000 nvme: count number of ignored interrupts Count the number of times we're asked to process completions, but that we ignore because the state of the qpair isn't in RECOVERY_NONE. Sponsored by: Netflix Reviewed by: mav, chuck Differential Revision: https://reviews.freebsd.org/D32212 (cherry picked from commit 587aa25525e54ea775298c402acd7a647f9838fb) --- sys/dev/nvme/nvme_private.h | 1 + sys/dev/nvme/nvme_qpair.c | 5 ++++- sys/dev/nvme/nvme_sysctl.c | 24 ++++++++++++++++++++++++ 3 files changed, 29 insertions(+), 1 deletion(-) diff --git a/sys/dev/nvme/nvme_private.h b/sys/dev/nvme/nvme_private.h index 02cecaf03a97..b00130910a82 100644 --- a/sys/dev/nvme/nvme_private.h +++ b/sys/dev/nvme/nvme_private.h @@ -195,6 +195,7 @@ struct nvme_qpair { int64_t num_intr_handler_calls; int64_t num_retries; int64_t num_failures; + int64_t num_ignored; struct nvme_command *cmd; struct nvme_completion *cpl; diff --git a/sys/dev/nvme/nvme_qpair.c b/sys/dev/nvme/nvme_qpair.c index 827054efd48e..788322092f88 100644 --- a/sys/dev/nvme/nvme_qpair.c +++ b/sys/dev/nvme/nvme_qpair.c @@ -544,8 +544,10 @@ nvme_qpair_process_completions(struct nvme_qpair *qpair) * as soon as initialization is complete and we start sending commands * to the device. */ - if (qpair->recovery_state != RECOVERY_NONE) + if (qpair->recovery_state != RECOVERY_NONE) { + qpair->num_ignored++; return (false); + } /* * Sanity check initialization. After we reset the hardware, the phase @@ -746,6 +748,7 @@ nvme_qpair_construct(struct nvme_qpair *qpair, qpair->num_intr_handler_calls = 0; qpair->num_retries = 0; qpair->num_failures = 0; + qpair->num_ignored = 0; qpair->cmd = (struct nvme_command *)queuemem; qpair->cpl = (struct nvme_completion *)(queuemem + cmdsz); prpmem = (uint8_t *)(queuemem + cmdsz + cplsz); diff --git a/sys/dev/nvme/nvme_sysctl.c b/sys/dev/nvme/nvme_sysctl.c index 1f1e1ce4a628..4367a21861b3 100644 --- a/sys/dev/nvme/nvme_sysctl.c +++ b/sys/dev/nvme/nvme_sysctl.c @@ -164,6 +164,7 @@ nvme_qpair_reset_stats(struct nvme_qpair *qpair) qpair->num_intr_handler_calls = 1; qpair->num_retries = 0; qpair->num_failures = 0; + qpair->num_ignored = 0; } static int @@ -226,6 +227,21 @@ nvme_sysctl_num_failures(SYSCTL_HANDLER_ARGS) return (sysctl_handle_64(oidp, &num_failures, 0, req)); } +static int +nvme_sysctl_num_ignored(SYSCTL_HANDLER_ARGS) +{ + struct nvme_controller *ctrlr = arg1; + int64_t num_ignored = 0; + int i; + + num_ignored = ctrlr->adminq.num_ignored; + + for (i = 0; i < ctrlr->num_io_queues; i++) + num_ignored += ctrlr->ioq[i].num_ignored; + + return (sysctl_handle_64(oidp, &num_ignored, 0, req)); +} + static int nvme_sysctl_reset_stats(SYSCTL_HANDLER_ARGS) { @@ -281,6 +297,9 @@ nvme_sysctl_initialize_queue(struct nvme_qpair *qpair, SYSCTL_ADD_QUAD(ctrlr_ctx, que_list, OID_AUTO, "num_failures", CTLFLAG_RD, &qpair->num_failures, "Number of commands ending in failure after all retries"); + SYSCTL_ADD_QUAD(ctrlr_ctx, que_list, OID_AUTO, "num_ignored", + CTLFLAG_RD, &qpair->num_ignored, + "Number of interrupts posted, but were administratively ignored"); SYSCTL_ADD_PROC(ctrlr_ctx, que_list, OID_AUTO, "dump_debug", CTLTYPE_UINT | CTLFLAG_RW | CTLFLAG_MPSAFE, @@ -344,6 +363,11 @@ nvme_sysctl_initialize_ctrlr(struct nvme_controller *ctrlr) ctrlr, 0, nvme_sysctl_num_failures, "IU", "Number of commands ending in failure after all retries"); + SYSCTL_ADD_PROC(ctrlr_ctx, ctrlr_list, OID_AUTO, + "num_ignored", CTLTYPE_S64 | CTLFLAG_RD | CTLFLAG_MPSAFE, + ctrlr, 0, nvme_sysctl_num_ignored, "IU", + "Number of interrupts ignored administratively"); + SYSCTL_ADD_PROC(ctrlr_ctx, ctrlr_list, OID_AUTO, "reset_stats", CTLTYPE_UINT | CTLFLAG_RW | CTLFLAG_MPSAFE, ctrlr, 0, nvme_sysctl_reset_stats, "IU", "Reset statistics to zero");