From nobody Thu Oct 29 23:37:33 2026 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jG0yQ1mwTz6xr5t for ; Thu, 29 Oct 2026 23:37:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jG0yP6q7Mz4t8f for ; Thu, 29 Oct 2026 23:37:33 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793317054; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=AamGnJQAe1HkD5ukcFFZ8Q/cfgaqgzvj2Fn6QA3sNcQ=; b=jgp/7mDZSM9W6X8jMK6dQkLKcVAmT4so31opBfAtZyd9TA45io7RPWJekO1kpj4emDIZch TfTeQJ8Go3KtzrI3VOs3l4q8FeaWr1rTjg+87PCaSBsBYhAhQgHEOuzV6emRluu2dsdbkv GOGQcarC7wZVG1A9iIVsnRqU3cVQPv04I1wyTk21pirPPDK0JW/0FZKmhVmybyfdbGkJaq YOoBU6lkvjulJPD2b+K1VeR/mIRH86mwsXw+98zyA/XPzk3jmz2+1u14aSbSh/Uhd+RzTU 7XQCnNnhHSbrmiT9Ka6zIhj3FUHy30i1eLBN5Tn4JziL5R5gweljRIRiGizjsA== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793317054; b=NReZTvZ1yibwWofDdkjDoUGNh3/y90QSqgf7m4+NvAyACtZ49SwO0SN1wMzQlkvHPE5/PT h65t+oShu4BrTzmv6QV9AgZ5Kp0bIjXSxeDiMkA8MOmkKNWSuWa7fPwzZO3N7jOF3DWh3m 8XUfDQX98tZ1i8Mz7hIMcmIuXBJMMZno2gTstvmxenBhNImA1eiq5mY6FnfpM9DOqoWhDV x1zAeFECqDV4uA1q9+IxXJM7VOGIf1LHhFkB9prmWlkzalhq37pR9LWGUhyrujv8DEYodY 3SGX4g1UfniKdkakIs4vw8TvgWi7OoEI5Cb4LMyNXKdevpei0+vKS09oPwc1SQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793317054; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=AamGnJQAe1HkD5ukcFFZ8Q/cfgaqgzvj2Fn6QA3sNcQ=; b=j8l/1C4YEcqCBF8fHEr2W/gla7kRsfi9lcfq5zUL2iCSlhAcGBSnQvc2H2tuqhmV1rIal7 gxkRLhXUlTBJRYybVwcnJskwnu0HasIuI6ed2DmtkGG431bFe5pXfA2sq+gSykeHfTSZvd y0h4+RpItR4NtWOw8IBuF8/Q9z/bBv4zDwvlxPV5PyScL02GI3uWUGKVfhsqOACZ4cnKDv 74892lL+/dqn8eoHBzcL8k0WeG10b5+EVqDDv2r0L7Mf+pI+sPI2lQYt8mGBWvAOVW/qwG XVUUjASS0AnH+dRWLYEEoNW+gxgx8XkcCg7b0vlsGoZ4TvW2iMi+4lklN5WZ8Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jG0yP6BLzz12Cs for ; Thu, 29 Oct 2026 23:37:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3ade8 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 29 Oct 2026 23:37:33 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Cc: Sumit Saxena From: Kevin Bowling Subject: git: c565e2223c46 - stable/15 - if_bnxt: support more Tx rings than Rx rings on P5+ List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org List-Id: Commits to the stable branches of the FreeBSD src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: c565e2223c46404744a3bebf0039a47d723653e8 Auto-Submitted: auto-generated Date: Thu, 29 Oct 2026 23:37:33 +0000 Message-Id: <6ae3d8bd.3ade8.7a8f688e@gitrepo.freebsd.org> The branch stable/15 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=c565e2223c46404744a3bebf0039a47d723653e8 commit c565e2223c46404744a3bebf0039a47d723653e8 Author: Sumit Saxena AuthorDate: 2026-08-01 08:52:51 +0000 Commit: Kevin Bowling CommitDate: 2026-10-29 23:25:51 +0000 if_bnxt: support more Tx rings than Rx rings on P5+ Add an unsupported-config guard rejecting nrxqsets > ntxqsets, and teach the P5+ NQ datapath to handle the opposite asymmetric case: extra Tx rings beyond the Rx ring count. softc->nq_rings[i].type now tags each NQ as SHARED_NQ (paired with an Rx CQ) or BNXT_TX_ONLY_NQ (dedicated to an extra Tx CQ with no backing Rx ring); bnxt_init() and bnxt_hwrm_resource_free() gate all Rx ring group/ring/stat-ctx alloc and free on IS_SHARED_NQ() so the extra Tx-only NQs don't touch nonexistent Rx ring state. Gate the new type-tagging and IS_SHARED_NQ checks on BNXT_CHIP_P5_PLUS(), since non-P5+ chips never allocate softc->nq_rings and would otherwise access a nonexistent array. BNXT_TX_ONLY_NQ is a distinct value from the existing MPC-private TX_CP_NQ; the two are unrelated and kept apart to avoid confusion. Suggested by: gallatin Reviewed by: gallatin Sponsored by: Broadcom Inc. Differential Revision: https://reviews.freebsd.org/D58618 (cherry picked from commit 644880d7b9b86fa16797fa45e7a876ac503227f9) --- sys/dev/bnxt/bnxt_en/bnxt.h | 11 +++ sys/dev/bnxt/bnxt_en/if_bnxt.c | 187 ++++++++++++++++++++++++----------------- 2 files changed, 121 insertions(+), 77 deletions(-) diff --git a/sys/dev/bnxt/bnxt_en/bnxt.h b/sys/dev/bnxt/bnxt_en/bnxt.h index 2cd4a77d406a..5de8244efb8d 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt.h +++ b/sys/dev/bnxt/bnxt_en/bnxt.h @@ -713,11 +713,22 @@ struct bnxt_cp_ring { #define Q_TYPE_TX 1 #define Q_TYPE_RX 2 #define TX_CP_NQ 3 /* MPC-private NQ ring (bnxt_mpc.c) */ +/* BNXT_TX_ONLY_NQ is kept distinct from the MPC-private TX_CP_NQ above - + * same field, unrelated arrays. */ +#define SHARED_NQ 4 +#define BNXT_TX_ONLY_NQ 5 +#define RX_CP_NQ 6 int msix_vec; /* MPC-private ring: bus_setup_intr vector */ struct resource *irq_res; /* MPC-private ring: bus_setup_intr resource */ void *irq_cookie; /* MPC-private ring: bus_setup_intr cookie */ }; +#define MAX_RXQ_INDEX(softc) ((softc)->nrxqsets - 1) +#define IS_SHARED_NQ(softc, i) \ + (BNXT_CHIP_P5_PLUS(softc) && (softc)->nq_rings[i].type == SHARED_NQ) +#define IS_TX_NQ(softc, i) ((softc)->nq_rings[i].type == BNXT_TX_ONLY_NQ) +#define IS_RX_NQ(softc, i) ((softc)->nq_rings[i].type == RX_CP_NQ) + struct bnxt_full_tpa_start { struct rx_tpa_start_cmpl low; struct rx_tpa_start_cmpl_hi high; diff --git a/sys/dev/bnxt/bnxt_en/if_bnxt.c b/sys/dev/bnxt/bnxt_en/if_bnxt.c index 0505295ac17b..1aae1271c24f 100644 --- a/sys/dev/bnxt/bnxt_en/if_bnxt.c +++ b/sys/dev/bnxt/bnxt_en/if_bnxt.c @@ -3008,6 +3008,19 @@ bnxt_attach_pre(if_ctx_t ctx) device_printf(softc->dev, "WARNING: ntxd0 (%d) should be at least 2 * ntxd1 (%d). Driver may be unstable\n", scctx->isc_ntxd[0], scctx->isc_ntxd[1]); + + /* + * Unsupported configuration: number of Rx rings + * exceeding number of Tx rings + */ + if (scctx->isc_nrxqsets > scctx->isc_ntxqsets) { + device_printf(softc->dev, "Unsupported config: number of" + " Rx rings(%u) exceeds number of Tx rings(%u)\n", + scctx->isc_nrxqsets, scctx->isc_ntxqsets); + rc = EOPNOTSUPP; + goto failed; + } + scctx->isc_txqsizes[0] = sizeof(struct cmpl_base) * scctx->isc_ntxd[0]; scctx->isc_txqsizes[1] = sizeof(struct tx_bd_short) * scctx->isc_ntxd[1]; @@ -3444,31 +3457,33 @@ bnxt_hwrm_resource_free(struct bnxt_softc *softc) if (rc) goto fail; - for (i = 0; i < softc->nrxqsets; i++) { - rc = bnxt_hwrm_ring_grp_free(softc, &softc->grp_info[i]); - if (rc) - goto fail; + for (i = 0; i < softc->ntxqsets; i++) { + if (IS_SHARED_NQ(softc, i) || !BNXT_CHIP_P5_PLUS(softc)) { + rc = bnxt_hwrm_ring_grp_free(softc, &softc->grp_info[i]); + if (rc) + goto fail; - rc = bnxt_hwrm_ring_free(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG, - &softc->ag_rings[i], - (uint16_t)HWRM_NA_SIGNATURE); - if (rc) - goto fail; + rc = bnxt_hwrm_ring_free(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG, + &softc->ag_rings[i], + (uint16_t)HWRM_NA_SIGNATURE); + if (rc) + goto fail; - rc = bnxt_hwrm_ring_free(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX, - &softc->rx_rings[i], - softc->rx_cp_rings[i].ring.phys_id); - if (rc) - goto fail; + rc = bnxt_hwrm_ring_free(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX, + &softc->rx_rings[i], + softc->rx_cp_rings[i].ring.phys_id); + if (rc) + goto fail; - rc = bnxt_hwrm_ring_free(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL, - &softc->rx_cp_rings[i].ring, - (uint16_t)HWRM_NA_SIGNATURE); - if (rc) - goto fail; + rc = bnxt_hwrm_ring_free(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL, + &softc->rx_cp_rings[i].ring, + (uint16_t)HWRM_NA_SIGNATURE); + if (rc) + goto fail; + } if (BNXT_CHIP_P5_PLUS(softc)) { rc = bnxt_hwrm_ring_free(softc, @@ -3479,9 +3494,11 @@ bnxt_hwrm_resource_free(struct bnxt_softc *softc) goto fail; } - rc = bnxt_hwrm_stat_ctx_free(softc, &softc->rx_cp_rings[i]); - if (rc) - goto fail; + if (IS_SHARED_NQ(softc, i) || !BNXT_CHIP_P5_PLUS(softc)) { + rc = bnxt_hwrm_stat_ctx_free(softc, &softc->rx_cp_rings[i]); + if (rc) + goto fail; + } } fail: @@ -3670,12 +3687,23 @@ skip_def_cp_ring: } } - for (i = 0; i < softc->nrxqsets; i++) { - /* Allocate the statistics context */ - rc = bnxt_hwrm_stat_ctx_alloc(softc, &softc->rx_cp_rings[i], - softc->rx_stats[i].idi_paddr); - if (rc) - goto fail; + for (i = 0; i < softc->ntxqsets; i++) { + /* Extra Tx rings over Rx: NQs up to MAX_RXQ_INDEX are shared + * Tx/Rx; NQs beyond that are Tx-only. */ + if (BNXT_CHIP_P5_PLUS(softc)) { + if (i <= MAX_RXQ_INDEX(softc)) + softc->nq_rings[i].type = SHARED_NQ; + else + softc->nq_rings[i].type = BNXT_TX_ONLY_NQ; + } + + if (IS_SHARED_NQ(softc, i) || !BNXT_CHIP_P5_PLUS(softc)) { + /* Allocate the statistics context */ + rc = bnxt_hwrm_stat_ctx_alloc(softc, &softc->rx_cp_rings[i], + softc->rx_stats[i].idi_paddr); + if (rc) + goto fail; + } if (BNXT_CHIP_P5_PLUS(softc)) { /* Allocate the NQ */ @@ -3694,53 +3722,56 @@ skip_def_cp_ring: softc->db_ops.bnxt_db_nq(&softc->nq_rings[i], 1); } - /* Allocate the completion ring */ - softc->rx_cp_rings[i].cons = UINT32_MAX; - softc->rx_cp_rings[i].raw_cons = UINT32_MAX; - softc->rx_cp_rings[i].v_bit = 1; - softc->rx_cp_rings[i].last_idx = UINT32_MAX; - softc->rx_cp_rings[i].toggle = 0; - bnxt_mark_cpr_invalid(&softc->rx_cp_rings[i]); - rc = bnxt_hwrm_ring_alloc(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL, - &softc->rx_cp_rings[i].ring); - bnxt_set_db_mask(softc, &softc->rx_cp_rings[i].ring, - HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL); - if (rc) - goto fail; - if (BNXT_CHIP_P5_PLUS(softc)) - softc->db_ops.bnxt_db_rx_cq(&softc->rx_cp_rings[i], 1); + if (IS_SHARED_NQ(softc, i) || !BNXT_CHIP_P5_PLUS(softc)) { + /* Allocate the completion ring */ + softc->rx_cp_rings[i].cons = UINT32_MAX; + softc->rx_cp_rings[i].raw_cons = UINT32_MAX; + softc->rx_cp_rings[i].v_bit = 1; + softc->rx_cp_rings[i].last_idx = UINT32_MAX; + softc->rx_cp_rings[i].toggle = 0; + bnxt_mark_cpr_invalid(&softc->rx_cp_rings[i]); + rc = bnxt_hwrm_ring_alloc(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL, + &softc->rx_cp_rings[i].ring); + bnxt_set_db_mask(softc, &softc->rx_cp_rings[i].ring, + HWRM_RING_ALLOC_INPUT_RING_TYPE_L2_CMPL); + if (rc) + goto fail; - /* Allocate the RX ring */ - rc = bnxt_hwrm_ring_alloc(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX, &softc->rx_rings[i]); - bnxt_set_db_mask(softc, &softc->rx_rings[i], - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX); - if (rc) - goto fail; - softc->db_ops.bnxt_db_rx(&softc->rx_rings[i], 0); + if (BNXT_CHIP_P5_PLUS(softc)) + softc->db_ops.bnxt_db_rx_cq(&softc->rx_cp_rings[i], 1); - /* Allocate the AG ring */ - rc = bnxt_hwrm_ring_alloc(softc, - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG, - &softc->ag_rings[i]); - bnxt_set_db_mask(softc, &softc->ag_rings[i], - HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG); - if (rc) - goto fail; - softc->db_ops.bnxt_db_rx(&softc->ag_rings[i], 0); + /* Allocate the RX ring */ + rc = bnxt_hwrm_ring_alloc(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX, &softc->rx_rings[i]); + bnxt_set_db_mask(softc, &softc->rx_rings[i], + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX); + if (rc) + goto fail; + softc->db_ops.bnxt_db_rx(&softc->rx_rings[i], 0); - /* Allocate the ring group */ - softc->grp_info[i].stats_ctx = - softc->rx_cp_rings[i].stats_ctx_id; - softc->grp_info[i].rx_ring_id = softc->rx_rings[i].phys_id; - softc->grp_info[i].ag_ring_id = softc->ag_rings[i].phys_id; - softc->grp_info[i].cp_ring_id = - softc->rx_cp_rings[i].ring.phys_id; - rc = bnxt_hwrm_ring_grp_alloc(softc, &softc->grp_info[i]); - if (rc) - goto fail; + /* Allocate the AG ring */ + rc = bnxt_hwrm_ring_alloc(softc, + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG, + &softc->ag_rings[i]); + bnxt_set_db_mask(softc, &softc->ag_rings[i], + HWRM_RING_ALLOC_INPUT_RING_TYPE_RX_AGG); + if (rc) + goto fail; + softc->db_ops.bnxt_db_rx(&softc->ag_rings[i], 0); + + /* Allocate the ring group */ + softc->grp_info[i].stats_ctx = + softc->rx_cp_rings[i].stats_ctx_id; + softc->grp_info[i].rx_ring_id = softc->rx_rings[i].phys_id; + softc->grp_info[i].ag_ring_id = softc->ag_rings[i].phys_id; + softc->grp_info[i].cp_ring_id = + softc->rx_cp_rings[i].ring.phys_id; + rc = bnxt_hwrm_ring_grp_alloc(softc, &softc->grp_info[i]); + if (rc) + goto fail; + } } /* Inform PF to approve MAC as default VF MAC. */ @@ -4395,10 +4426,12 @@ bnxt_tx_queue_intr_enable(if_ctx_t ctx, uint16_t qid) { struct bnxt_softc *softc = iflib_get_softc(ctx); - if (BNXT_CHIP_P5_PLUS(softc)) - softc->db_ops.bnxt_db_nq(&softc->nq_rings[qid], 1); - else + if (BNXT_CHIP_P5_PLUS(softc)) { + if (!IS_SHARED_NQ(softc, qid)) + softc->db_ops.bnxt_db_nq(&softc->nq_rings[qid], 1); + } else { softc->db_ops.bnxt_db_rx_cq(&softc->tx_cp_rings[qid], 1); + } return 0; }