From nobody Fri Oct 09 10:12:54 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4j1N3B6clBz6vRZN for ; Fri, 09 Oct 2026 10:12:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4j1N3B5540z4Y7n for ; Fri, 09 Oct 2026 10:12:54 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1791540774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=pLY2Np5yPRaQjUPKwmvg3ngud8KQBUrVTupjdqxeZXU=; b=vqA9uLNmQK1W64NdJged66A9tlHn6WpitLqGwJFOtYI4alUlyw+qncu+8ZGjl8MhAoyuLN /l/gfl46mVan3NSwMQNNxJHImdAD2+uujincrMveHs1dxo1nojzPJI6OBoOQ4aBOumzEKl TX/WBCqdAdDWdp5Bb7gwNa6onkVTRTVOxw72AnUTfOwlCsZ6JJhwI4RsQgfTw8MM25ZmQM Sc1ZGzVsWu7hOzrEhXb4IW5XvoazGK9mUEeoT6UDnLTWIQpUljT/jfVWS9rI6C2MN0n33B YojfDYbbw9OnaU1r7csnlRUUma5esfrl3aXNmq+YlpOCFNR1LBqx395UC8uOjw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1791540774; b=CeZ6EwPpz3S1n1Z4L6go7wsLHQmcFO1LAb0sH+c9uI6qsjiMBHFCREgxP8551dkxhqITr7 FodLSoBSGUUwHeNBuvnUqu2Oceh4Rz2a8M+dq2CTlL6SVHzEJbibXT7Z2R1Peh21FZgC1a XA135GApFVFW4xhTN3IfaikRhwaojUYfgAP1MsOGt7Jna9LON1qtcJyi7xIM0lAi9O8XIS 8oAJ7tflg4R2MNzY92/LZXHeJRAoBWX611eJWfuwYwHt6UnUah4DbneLtCbxbnAu0pNfSB WY1kiHQ8nPIprh7+3QrHx/GE9SVflWi+fOm/tZgKwAaX0QSZdO6+eLz15PkkZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1791540774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=pLY2Np5yPRaQjUPKwmvg3ngud8KQBUrVTupjdqxeZXU=; b=DpvRbz5iVfOPLKH8z0EDwO8jTjR1X7U+DVNx+KpvGFPBz8jkXkGxmKz8VXFSCrKX/EKOBv xRyzAGfbybYAp+DKetSiw7QEMpHLh5LlQQ57ZhKI0OtCkrbvUxh6uBa1BuUzh6nDskuTHu pBTXttLgF0qBTvWV21K+gpWRGBxEMjAL59ShKFGFUKUB8pVmaaJeaKv/vETJucQeaSRBqN ZKwgmHTo2YNFynzq2g6b8zVMTiDKLfhKHZwooSVbDIrN3W85OFtxfojOHZMCav/z9BVCg0 LWy0rmm9f/52uAJy4go64qohMvf6JovvElT2aSt+gNj2RqJKvRbF0Mc9vv4blg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4j1N3B464tz1LWw for ; Fri, 09 Oct 2026 10:12:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 43416 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 09 Oct 2026 10:12:54 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Sumit Saxena From: Sumit Saxena Subject: git: 644880d7b9b8 - main - if_bnxt: support more Tx rings than Rx rings on P5+ List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ssaxena X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 644880d7b9b86fa16797fa45e7a876ac503227f9 Auto-Submitted: auto-generated Date: Fri, 09 Oct 2026 10:12:54 +0000 Message-Id: <6ac8be26.43416.31f33a4a@gitrepo.freebsd.org> The branch main has been updated by ssaxena: URL: https://cgit.FreeBSD.org/src/commit/?id=644880d7b9b86fa16797fa45e7a876ac503227f9 commit 644880d7b9b86fa16797fa45e7a876ac503227f9 Author: Sumit Saxena AuthorDate: 2026-08-01 08:52:51 +0000 Commit: Sumit Saxena CommitDate: 2026-10-09 08:24:45 +0000 if_bnxt: support more Tx rings than Rx rings on P5+ Add an unsupported-config guard rejecting nrxqsets > ntxqsets, and teach the P5+ NQ datapath to handle the opposite asymmetric case: extra Tx rings beyond the Rx ring count. softc->nq_rings[i].type now tags each NQ as SHARED_NQ (paired with an Rx CQ) or BNXT_TX_ONLY_NQ (dedicated to an extra Tx CQ with no backing Rx ring); bnxt_init() and bnxt_hwrm_resource_free() gate all Rx ring group/ring/stat-ctx alloc and free on IS_SHARED_NQ() so the extra Tx-only NQs don't touch nonexistent Rx ring state. Gate the new type-tagging and IS_SHARED_NQ checks on BNXT_CHIP_P5_PLUS(), since non-P5+ chips never allocate softc->nq_rings and would otherwise access a nonexistent array. BNXT_TX_ONLY_NQ is a distinct value from the existing MPC-private TX_CP_NQ; the two are unrelated and kept apart to avoid confusion. Suggested by: gallatin Reviewed by: gallatin MFC after: 2 weeks Sponsored by: Broadcom Inc. Differential Revision: https://reviews.freebsd.org/D58618 --- sys/dev/bnxt/bnxt_en/bnxt.h | 11 +++ sys/dev/bnxt/bnxt_en/if_bnxt.c | 187 ++++++++++++++++++++++++----------------- 2 files changed, 121 insertions(+), 77 deletions(-) diff --git a/sys/dev/bnxt/bnxt_en/bnxt.h b/sys/dev/bnxt/bnxt_en/bnxt.h index 37914dae9a1d..14f17b6d288a 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt.h +++ b/sys/dev/bnxt/bnxt_en/bnxt.h @@ -712,11 +712,22 @@ struct bnxt_cp_ring { #define Q_TYPE_TX 1 #define Q_TYPE_RX 2 #define TX_CP_NQ 3 /* MPC-private NQ ring (bnxt_mpc.c) */ +/* BNXT_TX_ONLY_NQ is kept distinct from the MPC-private TX_CP_NQ above - + * same field, unrelated arrays. */ +#define SHARED_NQ 4 +#define BNXT_TX_ONLY_NQ 5 +#define RX_CP_NQ 6 int msix_vec; /* MPC-private ring: bus_setup_intr vector */ struct resource *irq_res; /* MPC-private ring: bus_setup_intr resource */ void *irq_cookie; /* MPC-private ring: bus_setup_intr cookie */ }; +#define MAX_RXQ_INDEX(softc) ((softc)->nrxqsets - 1) +#define IS_SHARED_NQ(softc, i) \ + (BNXT_CHIP_P5_PLUS(softc) && (softc)->nq_rings[i].type == SHARED_NQ) +#define IS_TX_NQ(softc, i) ((softc)->nq_rings[i].type == BNXT_TX_ONLY_NQ) +#define IS_RX_NQ(softc, i) ((softc)->nq_rings[i].type == RX_CP_NQ) + struct bnxt_full_tpa_start { struct rx_tpa_start_cmpl low; struct rx_tpa_start_cmpl_hi high; diff --git a/sys/dev/bnxt/bnxt_en/if_bnxt.c b/sys/dev/bnxt/bnxt_en/if_bnxt.c index b7dd72ffb177..2129c0ff51b0 100644 --- a/sys/dev/bnxt/bnxt_en/if_bnxt.c +++ b/sys/dev/bnxt/bnxt_en/if_bnxt.c @@ -3000,6 +3000,19 @@ bnxt_attach_pre(if_ctx_t ctx) device_printf(softc->dev, "WARNING: ntxd0 (%d) should be at least 2 * ntxd1 (%d). Driver may be unstable\n", scctx->isc_ntxd[0], scctx->isc_ntxd[1]); + + /* + * Unsupported configuration: number of Rx rings + * exceeding number of Tx rings + */ + if (scctx->isc_nrxqsets > scctx->isc_ntxqsets) { + device_printf(softc->dev, "Unsupported config: number of" + " Rx rings(%u) exceeds number of Tx rings(%u)\n", + scctx->isc_nrxqsets, scctx->isc_ntxqsets); + rc = EOPNOTSUPP; + goto failed; + } + scctx->isc_txqsizes[0] = sizeof(struct cmpl_base) * scctx->isc_ntxd[0]; scctx->isc_txqsizes[1] = sizeof(struct tx_bd_short) * scctx->isc_ntxd[1]; @@ -3436,31 +3449,33 @@ bnxt_hwrm_resource_free(struct bnxt_softc *softc) if (rc) goto fail; - for (i = 0; i < softc->nrxqsets; i++) { - rc = bnxt_hwrm_ring_grp_free(softc, &softc->grp_info[i]); - if (rc) - goto fail; + for (i = 0; i < softc->ntxqsets; i++) { + if (IS_SHARED_NQ(softc, i) || !BNXT_CHIP_P5_PLUS(softc)) { + rc = bnxt_hwrm_ring_grp_free(softc, &softc->grp_info[i]); + if (rc) + goto fail; - rc = bnxt_hwrm_ring_free(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG, - &softc->ag_rings[i], - (uint16_t)HWRM_NA_SIGNATURE); - if (rc) - goto fail; + rc = bnxt_hwrm_ring_free(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG, + &softc->ag_rings[i], + (uint16_t)HWRM_NA_SIGNATURE); + if (rc) + goto fail; - rc = bnxt_hwrm_ring_free(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX, - &softc->rx_rings[i], - softc->rx_cp_rings[i].ring.phys_id); - if (rc) - goto fail; + rc = bnxt_hwrm_ring_free(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX, + &softc->rx_rings[i], + softc->rx_cp_rings[i].ring.phys_id); + if (rc) + goto fail; - rc = bnxt_hwrm_ring_free(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL, - &softc->rx_cp_rings[i].ring, - (uint16_t)HWRM_NA_SIGNATURE); - if (rc) - goto fail; + rc = bnxt_hwrm_ring_free(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL, + &softc->rx_cp_rings[i].ring, + (uint16_t)HWRM_NA_SIGNATURE); + if (rc) + goto fail; + } if (BNXT_CHIP_P5_PLUS(softc)) { rc = bnxt_hwrm_ring_free(softc, @@ -3471,9 +3486,11 @@ bnxt_hwrm_resource_free(struct bnxt_softc *softc) goto fail; } - rc = bnxt_hwrm_stat_ctx_free(softc, &softc->rx_cp_rings[i]); - if (rc) - goto fail; + if (IS_SHARED_NQ(softc, i) || !BNXT_CHIP_P5_PLUS(softc)) { + rc = bnxt_hwrm_stat_ctx_free(softc, &softc->rx_cp_rings[i]); + if (rc) + goto fail; + } } fail: @@ -3662,12 +3679,23 @@ skip_def_cp_ring: } } - for (i = 0; i < softc->nrxqsets; i++) { - /* Allocate the statistics context */ - rc = bnxt_hwrm_stat_ctx_alloc(softc, &softc->rx_cp_rings[i], - softc->rx_stats[i].idi_paddr); - if (rc) - goto fail; + for (i = 0; i < softc->ntxqsets; i++) { + /* Extra Tx rings over Rx: NQs up to MAX_RXQ_INDEX are shared + * Tx/Rx; NQs beyond that are Tx-only. */ + if (BNXT_CHIP_P5_PLUS(softc)) { + if (i <= MAX_RXQ_INDEX(softc)) + softc->nq_rings[i].type = SHARED_NQ; + else + softc->nq_rings[i].type = BNXT_TX_ONLY_NQ; + } + + if (IS_SHARED_NQ(softc, i) || !BNXT_CHIP_P5_PLUS(softc)) { + /* Allocate the statistics context */ + rc = bnxt_hwrm_stat_ctx_alloc(softc, &softc->rx_cp_rings[i], + softc->rx_stats[i].idi_paddr); + if (rc) + goto fail; + } if (BNXT_CHIP_P5_PLUS(softc)) { /* Allocate the NQ */ @@ -3686,53 +3714,56 @@ skip_def_cp_ring: softc->db_ops.bnxt_db_nq(&softc->nq_rings[i], 1); } - /* Allocate the completion ring */ - softc->rx_cp_rings[i].cons = UINT32_MAX; - softc->rx_cp_rings[i].raw_cons = UINT32_MAX; - softc->rx_cp_rings[i].v_bit = 1; - softc->rx_cp_rings[i].last_idx = UINT32_MAX; - softc->rx_cp_rings[i].toggle = 0; - bnxt_mark_cpr_invalid(&softc->rx_cp_rings[i]); - rc = bnxt_hwrm_ring_alloc(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL, - &softc->rx_cp_rings[i].ring); - bnxt_set_db_mask(softc, &softc->rx_cp_rings[i].ring, - HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL); - if (rc) - goto fail; - if (BNXT_CHIP_P5_PLUS(softc)) - softc->db_ops.bnxt_db_rx_cq(&softc->rx_cp_rings[i], 1); + if (IS_SHARED_NQ(softc, i) || !BNXT_CHIP_P5_PLUS(softc)) { + /* Allocate the completion ring */ + softc->rx_cp_rings[i].cons = UINT32_MAX; + softc->rx_cp_rings[i].raw_cons = UINT32_MAX; + softc->rx_cp_rings[i].v_bit = 1; + softc->rx_cp_rings[i].last_idx = UINT32_MAX; + softc->rx_cp_rings[i].toggle = 0; + bnxt_mark_cpr_invalid(&softc->rx_cp_rings[i]); + rc = bnxt_hwrm_ring_alloc(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL, + &softc->rx_cp_rings[i].ring); + bnxt_set_db_mask(softc, &softc->rx_cp_rings[i].ring, + HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL); + if (rc) + goto fail; - /* Allocate the RX ring */ - rc = bnxt_hwrm_ring_alloc(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX, &softc->rx_rings[i]); - bnxt_set_db_mask(softc, &softc->rx_rings[i], - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX); - if (rc) - goto fail; - softc->db_ops.bnxt_db_rx(&softc->rx_rings[i], 0); + if (BNXT_CHIP_P5_PLUS(softc)) + softc->db_ops.bnxt_db_rx_cq(&softc->rx_cp_rings[i], 1); - /* Allocate the AG ring */ - rc = bnxt_hwrm_ring_alloc(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG, - &softc->ag_rings[i]); - bnxt_set_db_mask(softc, &softc->ag_rings[i], - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG); - if (rc) - goto fail; - softc->db_ops.bnxt_db_rx(&softc->ag_rings[i], 0); + /* Allocate the RX ring */ + rc = bnxt_hwrm_ring_alloc(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX, &softc->rx_rings[i]); + bnxt_set_db_mask(softc, &softc->rx_rings[i], + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX); + if (rc) + goto fail; + softc->db_ops.bnxt_db_rx(&softc->rx_rings[i], 0); - /* Allocate the ring group */ - softc->grp_info[i].stats_ctx = - softc->rx_cp_rings[i].stats_ctx_id; - softc->grp_info[i].rx_ring_id = softc->rx_rings[i].phys_id; - softc->grp_info[i].ag_ring_id = softc->ag_rings[i].phys_id; - softc->grp_info[i].cp_ring_id = - softc->rx_cp_rings[i].ring.phys_id; - rc = bnxt_hwrm_ring_grp_alloc(softc, &softc->grp_info[i]); - if (rc) - goto fail; + /* Allocate the AG ring */ + rc = bnxt_hwrm_ring_alloc(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG, + &softc->ag_rings[i]); + bnxt_set_db_mask(softc, &softc->ag_rings[i], + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG); + if (rc) + goto fail; + softc->db_ops.bnxt_db_rx(&softc->ag_rings[i], 0); + + /* Allocate the ring group */ + softc->grp_info[i].stats_ctx = + softc->rx_cp_rings[i].stats_ctx_id; + softc->grp_info[i].rx_ring_id = softc->rx_rings[i].phys_id; + softc->grp_info[i].ag_ring_id = softc->ag_rings[i].phys_id; + softc->grp_info[i].cp_ring_id = + softc->rx_cp_rings[i].ring.phys_id; + rc = bnxt_hwrm_ring_grp_alloc(softc, &softc->grp_info[i]); + if (rc) + goto fail; + } } /* Inform PF to approve MAC as default VF MAC. */ @@ -4387,10 +4418,12 @@ bnxt_tx_queue_intr_enable(if_ctx_t ctx, uint16_t qid) { struct bnxt_softc *softc = iflib_get_softc(ctx); - if (BNXT_CHIP_P5_PLUS(softc)) - softc->db_ops.bnxt_db_nq(&softc->nq_rings[qid], 1); - else + if (BNXT_CHIP_P5_PLUS(softc)) { + if (!IS_SHARED_NQ(softc, qid)) + softc->db_ops.bnxt_db_nq(&softc->nq_rings[qid], 1); + } else { softc->db_ops.bnxt_db_rx_cq(&softc->tx_cp_rings[qid], 1); + } return 0; }