From nobody Fri Oct 09 10:12:50 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4j1N394MDWz6vRVF for ; Fri, 09 Oct 2026 10:12:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4j1N380Srcz4Y9t for ; Fri, 09 Oct 2026 10:12:52 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1791540772; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0sNb2QWM4WuBcUjFxZLaVplEgND0PLTQ+UTurOs2xq4=; b=KBWU0pQkZ8TLTztL7el2G/BTjvEsLtkl311D272/YN6yB7BPEWvoUFziTpbNH99omT7TAX 57LJgS0pBQ/qAEvhynX69NF97rpilDXIM48jo5u4losLeaOHG7nnY5ceGobVxbCadO+JKI V9CZlTZttyTjG1WCVk0o3KsKvXX3JFMiCa4rollXjhacc9oNNY8+Qi84coNrokCfmn3c6p PBkHN3JSGG09gw4uPLNGcqlAtjSC8dppA/UOhX0BzFQX8KF60yOGLtR9OfHQCWB2bBWdyE C5QIlnEfIC1/JIvvwDpWsVB5LXglsZNWf95N2ElAsul69D2I2fBGQDZGV3+hdg== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1791540772; b=osc5yvmwQ3OxM83c8ZKJweSvU664ScSXrgafwEj4AvBn1h345MmhXSe5U1MZzP8kUABLtX 4PM2A2L+hs0Qev1Xa4WMJPm19D73hjqcZo9aPCjgVZ7CsvXGajCFj0zy0LCUxncF/9IsZi N7vrUC8a0P9hR9bQZvD1y9Ue+YFImvOlMndILNm8IrXBYrn7H7VLAMcRonKzjQh5+jU0J9 KTruWLvwFg1ojEQdF2JahBw9aH0lE24Mx8gXqiRymy2+285uis7vXAvGMW+MbWd8m9mzKG SOiESzP1duBB3cdP6jamvNh8h5szdP6B6xQCQf52X8G+WshVlHWH5ZKXw+tRpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1791540772; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0sNb2QWM4WuBcUjFxZLaVplEgND0PLTQ+UTurOs2xq4=; b=CXitcg7+sG8dxoHzTIID3/vDweMGFGjGi4ZhQZKtY/by8+hOyfHaMgtmunWaJqGF4fVLnc mGTr34tj+lE87AzL2epV1B8C1uje28S2AlmZFJw1wFWxNSVHsQBSfewu5iniwWAsl1MOgk 1XAmsI6h3cqtYSsqYd/wZNxdQXJpAqMXvBK8pnGOmq/gcGn1VcaKsKUyG/PZYBCaPbiiLZ USlEGMYdwJUzWi657DrERhXLDWZGH5JSdJIyiOKskHv1VoT5eb/99rVAmIajnheVpw3dvC VV9W7GYo4ZZMsxpmOKmp9S8KAhOr9FafWZ+ZGTAUusgRypt/al5ShbgFCLZi9Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4j1N361dpKz1LTS for ; Fri, 09 Oct 2026 10:12:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 41ac9 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 09 Oct 2026 10:12:50 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Sumit Saxena From: Sumit Saxena Subject: git: d1619865da1c - main - if_bnxt: add support for HW generic stats List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ssaxena X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: d1619865da1cd4c5b557158b4bbf666e0f7f040b Auto-Submitted: auto-generated Date: Fri, 09 Oct 2026 10:12:50 +0000 Message-Id: <6ac8be22.41ac9.6f8383f6@gitrepo.freebsd.org> The branch main has been updated by ssaxena: URL: https://cgit.FreeBSD.org/src/commit/?id=d1619865da1cd4c5b557158b4bbf666e0f7f040b commit d1619865da1cd4c5b557158b4bbf666e0f7f040b Author: Sumit Saxena AuthorDate: 2026-07-31 16:00:58 +0000 Commit: Sumit Saxena CommitDate: 2026-10-09 08:24:44 +0000 if_bnxt: add support for HW generic stats HW generic stats provide information about PCIe transactions and out-of-order doorbell writes, useful for debug. Allocate a DMA buffer for them (disabling BNXT_FW_CAP_GENERIC_STATS if the allocation fails), fetch them via HWRM_STAT_GENERIC_QSTATS once per admin-status tick on P5+ chips, and expose them to userspace under dev.bnxt..hwstats.generic_stats.* via sysctl. Reviewed by: gallatin MFC after: 2 weeks Sponsored by: Broadcom Inc. Differential Revision: https://reviews.freebsd.org/D58614 --- sys/dev/bnxt/bnxt_en/bnxt.h | 3 ++ sys/dev/bnxt/bnxt_en/bnxt_hwrm.c | 20 ++++++++ sys/dev/bnxt/bnxt_en/bnxt_hwrm.h | 1 + sys/dev/bnxt/bnxt_en/bnxt_sysctl.c | 94 ++++++++++++++++++++++++++++++++++++++ sys/dev/bnxt/bnxt_en/bnxt_sysctl.h | 1 + sys/dev/bnxt/bnxt_en/if_bnxt.c | 15 ++++++ 6 files changed, 134 insertions(+) diff --git a/sys/dev/bnxt/bnxt_en/bnxt.h b/sys/dev/bnxt/bnxt_en/bnxt.h index 09092cf7d59e..763748974458 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt.h +++ b/sys/dev/bnxt/bnxt_en/bnxt.h @@ -1294,6 +1294,9 @@ struct bnxt_softc { struct tx_port_stats_ext *tx_port_stats_ext; struct rx_port_stats_ext *rx_port_stats_ext; + struct iflib_dma_info hw_generic_stats; + struct generic_sw_hw_stats *generic_stats; + uint16_t fw_rx_stats_ext_size; uint16_t fw_tx_stats_ext_size; uint16_t hw_ring_stats_size; diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c index 680bb593c4b4..73afb62b21ad 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c @@ -2230,6 +2230,26 @@ bnxt_hwrm_port_qstats(struct bnxt_softc *softc) return rc; } + +int bnxt_hwrm_generic_qstats(struct bnxt_softc *bp, u8 flags) +{ + struct hwrm_stat_generic_qstats_input req = {0}; + int rc; + + if (!(bp->fw_cap & BNXT_FW_CAP_GENERIC_STATS)) + return 0; + + bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_STAT_GENERIC_QSTATS); + + req.flags = flags; + req.generic_stat_size = sizeof(struct generic_sw_hw_stats); + req.generic_stat_host_addr = htole64(bp->hw_generic_stats.idi_paddr); + + rc = hwrm_send_message(bp, &req, sizeof(req)); + + return rc; +} + static int bnxt_hwrm_pri2cos_idx(struct bnxt_softc *softc, uint32_t path_dir) { struct hwrm_queue_pri2cos_qcfg_input req = {0}; diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h index 641a1e9ebe3a..e507b716303d 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h @@ -68,6 +68,7 @@ int bnxt_hwrm_stat_ctx_alloc(struct bnxt_softc *softc, struct bnxt_cp_ring *cpr, uint64_t paddr); int bnxt_hwrm_stat_ctx_free(struct bnxt_softc *softc, struct bnxt_cp_ring *cpr); int bnxt_hwrm_port_qstats(struct bnxt_softc *softc); +int bnxt_hwrm_generic_qstats(struct bnxt_softc *softc, uint8_t flags); int bnxt_hwrm_port_qstats_ext(struct bnxt_softc *softc); int bnxt_hwrm_port_led_qcaps(struct bnxt_softc *softc); int bnxt_hwrm_port_led_cfg(struct bnxt_softc *softc, bool led_on); diff --git a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c index fad3982f5581..37a878c9d85f 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c @@ -336,6 +336,100 @@ bnxt_create_tx_sysctls(struct bnxt_softc *softc, int txr) return 0; } +int +bnxt_create_generic_stats_sysctls(struct bnxt_softc *softc) +{ + struct sysctl_oid *oid; + char name[32]; + char desc[64]; + + sprintf(name, "generic_stats"); + sprintf(desc, "Generic Stats"); + oid = SYSCTL_ADD_NODE(&softc->hw_stats, + SYSCTL_CHILDREN(softc->hw_stats_oid), OID_AUTO, name, + CTLFLAG_RD | CTLFLAG_MPSAFE, 0, desc); + if (!oid) + return ENOMEM; + + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_statistics_tx_tlp", CTLFLAG_RD, + &softc->generic_stats->pcie_statistics_tx_tlp, "pcie_statistics_tx_tlp"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_statistics_rx_tlp", CTLFLAG_RD, + &softc->generic_stats->pcie_statistics_rx_tlp, "pcie_statistics_rx_tlp"); + + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_hdr_posted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_hdr_posted, "pcie_credit_fc_hdr_posted"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_hdr_nonposted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_hdr_nonposted, "pcie_credit_fc_hdr_nonposted"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_hdr_cmpl", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_hdr_cmpl, "pcie_credit_fc_hdr_cmpl"); + + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_data_posted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_data_posted, "pcie_credit_fc_data_posted"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_data_nonposted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_data_nonposted, "pcie_credit_fc_data_nonposted"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_data_cmpl", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_data_cmpl, "pcie_credit_fc_data_cmpl"); + + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_tgt_nonposted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_tgt_nonposted, "pcie_credit_fc_tgt_nonposted"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_tgt_data_posted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_tgt_data_posted, "pcie_credit_fc_tgt_data_posted"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_tgt_hdr_posted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_tgt_hdr_posted, "pcie_credit_fc_tgt_hdr_posted"); + + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_cmpl_hdr_posted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_cmpl_hdr_posted, "pcie_credit_fc_cmpl_hdr_posted"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_credit_fc_cmpl_data_posted", CTLFLAG_RD, + &softc->generic_stats->pcie_credit_fc_cmpl_data_posted, "pcie_credit_fc_cmpl_data_posted"); + + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_cmpl_longest", CTLFLAG_RD, + &softc->generic_stats->pcie_cmpl_longest, "pcie_cmpl_longest"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "pcie_cmpl_shortest", CTLFLAG_RD, + &softc->generic_stats->pcie_cmpl_shortest, "pcie_cmpl_shortest"); + + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "cache_miss_count_cfcq", CTLFLAG_RD, + &softc->generic_stats->cache_miss_count_cfcq, "cache_miss_count_cfcq"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "cache_miss_count_cfcs", CTLFLAG_RD, + &softc->generic_stats->cache_miss_count_cfcs, "cache_miss_count_cfcs"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "cache_miss_count_cfcc", CTLFLAG_RD, + &softc->generic_stats->cache_miss_count_cfcc, "cache_miss_count_cfcc"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "cache_miss_count_cfcm", CTLFLAG_RD, + &softc->generic_stats->cache_miss_count_cfcm, "cache_miss_count_cfcm"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "hw_db_recov_dbs_dropped", CTLFLAG_RD, + &softc->generic_stats->hw_db_recov_dbs_dropped, "hw_db_recov_dbs_dropped"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "hw_db_recov_drops_serviced", CTLFLAG_RD, + &softc->generic_stats->hw_db_recov_drops_serviced, "hw_db_recov_drops_serviced"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "hw_db_recov_dbs_recovered", CTLFLAG_RD, + &softc->generic_stats->hw_db_recov_dbs_recovered, "hw_db_recov_dbs_recovered"); + SYSCTL_ADD_QUAD(&softc->hw_stats, SYSCTL_CHILDREN(oid), OID_AUTO, + "hw_db_recov_oo_drop_count", CTLFLAG_RD, + &softc->generic_stats->hw_db_recov_oo_drop_count, "hw_db_recov_oo_drop_count"); + + return 0; +} + int bnxt_create_port_stats_sysctls(struct bnxt_softc *softc) { diff --git a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.h b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.h index 317cf06ae4b2..2a4531c69779 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.h +++ b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.h @@ -32,6 +32,7 @@ int bnxt_init_sysctl_ctx(struct bnxt_softc *softc); int bnxt_free_sysctl_ctx(struct bnxt_softc *softc); int bnxt_create_port_stats_sysctls(struct bnxt_softc *softc); +int bnxt_create_generic_stats_sysctls(struct bnxt_softc *softc); int bnxt_create_tx_sysctls(struct bnxt_softc *softc, int txr); int bnxt_create_rx_sysctls(struct bnxt_softc *softc, int rxr); int bnxt_create_ver_sysctls(struct bnxt_softc *softc); diff --git a/sys/dev/bnxt/bnxt_en/if_bnxt.c b/sys/dev/bnxt/bnxt_en/if_bnxt.c index 6566c1e8f442..e0e606189fdb 100644 --- a/sys/dev/bnxt/bnxt_en/if_bnxt.c +++ b/sys/dev/bnxt/bnxt_en/if_bnxt.c @@ -735,6 +735,7 @@ bnxt_queues_free(if_ctx_t ctx) // Free RX queues for (i=0; inrxqsets; i++) iflib_dma_free(&softc->rx_stats[i]); + iflib_dma_free(&softc->hw_generic_stats); iflib_dma_free(&softc->hw_tx_port_stats); iflib_dma_free(&softc->hw_rx_port_stats); iflib_dma_free(&softc->hw_tx_port_stats_ext); @@ -808,6 +809,17 @@ bnxt_rx_queues_alloc(if_ctx_t ctx, caddr_t *vaddrs, */ #define BNXT_PORT_STAT_PADDING 512 + rc = iflib_dma_alloc(ctx, sizeof(struct generic_sw_hw_stats), &softc->hw_generic_stats, 0); + /* Generic stats are optional */ + if (rc) { + softc->fw_cap &= ~BNXT_FW_CAP_GENERIC_STATS; + iflib_dma_free(&softc->hw_generic_stats); + } else { + bus_dmamap_sync(softc->hw_generic_stats.idi_tag, + softc->hw_generic_stats.idi_map, BUS_DMASYNC_PREREAD); + softc->generic_stats = (void *) softc->hw_generic_stats.idi_vaddr; + } + rc = iflib_dma_alloc(ctx, sizeof(struct rx_port_stats) + BNXT_PORT_STAT_PADDING, &softc->hw_rx_port_stats, 0); if (rc) @@ -916,6 +928,7 @@ bnxt_rx_queues_alloc(if_ctx_t ctx, caddr_t *vaddrs, bnxt_create_rx_sysctls(softc, i); } + bnxt_create_generic_stats_sysctls(softc); /* * When SR-IOV is enabled, avoid each VF sending PORT_QSTATS * HWRM every sec with which firmware timeouts can happen @@ -4256,6 +4269,8 @@ bnxt_update_admin_status(if_ctx_t ctx) bit_clear(softc->state_bv, BNXT_STATE_LINK_CHANGE); bnxt_media_status(softc->ctx, &ifmr); } + + bnxt_hwrm_generic_qstats(softc, 0); } return;