From nobody Mon Nov 02 07:56:45 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jJ3v16DHgz6xnDd for ; Mon, 02 Nov 2026 07:56:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jJ3v13kKFz4Bq1 for ; Mon, 02 Nov 2026 07:56:45 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793606205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=em/pFFO0a1hPaticPzHnB9GQgEwLIXxeF7Q+t/FLNqE=; b=dQHy6fdL2menq/8bB3mTPk9Hu3K9cB4bsZJFvIFpL4JhRHCUbT5VVFQ+DvY4IzEP9sy+al RKxZRNJbpVOMJA78DhM2QsGrRwDhSIPSBK5wCvq1P1glMBbgOsVckLtjLOCky8wWv3jfAh M+uPtbuhbljA82Yy8HC57wyzz8RoujsW7JvpqL+AW5xMRbsVI9BfOcVWm3PbgeZRRhIgtx 7G8bt3CO3/wNxgkROHGrrjn2Rx2VGkiJ98aoCLiOC+nsZ4zGxIr/b3sDxs0bcqfmI1Flyo s1B6YR4DvEz/v0bC5e5+oZuh5ZtaTcP6+a5pSqLgCivxqh/QTx/Fu/g+VVPdMA== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793606205; b=X7wU09WO9okEeAm+dajybA7c+Bh7CTFQ+qBg7/0GfN2QcTOSKUwBH5ZYbTqnaEjrnSmJuN ZOy5IuJFv7Y6ux2M29DZITkByuLNon8HByDzC7/Xi1i4r/5Bqb8VBjpReav4Ng6qJ+xrD2 nZOEFsDXlxMAyBAVips80drQ6B0y7rkDimM8QbKZvU9IfOQXl/JUHM614Z8ZVQzD8zop/i k+GCXBRuOzIs1FZyq87xmQxQEa5jqbCZ3l46qbc6t9AYP7afAah/jlosEDudaVm5r4DFyw VFAtVLTJFFkJVT7Y+gbzFlaHUJOdddUVgLHqvRvP2NhIon5hKS4HflTtusH0Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793606205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=em/pFFO0a1hPaticPzHnB9GQgEwLIXxeF7Q+t/FLNqE=; b=aGSHzl8Vd/ep+us+WbbereC9xF6ieLyvUVRvjJJdHLerUQPPdnDbJ34LaqfRdoh6bDFUnr E84TlCxZteFwp8uk7+ESBiwlg7lmRHfrKjjPhG6KFSycITHLUoIs8K9EAofnLXJp+E+4D0 ih2oc3Mmjz2pngYvKvG4CZgpsAcqYncMwNEhqnbjdCsn8yTxgxhCzBInKQl5BiB/Z738TQ vHI2Kt4UP4yhtrlL/3h8SwEd2aWdgFUxN+q6VSlaBHk/UvPyIjKBhnBwEnJH2OO7h/MDiM s6r9u8WvYDI/JLPlUzNkaf0FqH5blGz8KiVt32TDcq3W2PzOTfAaU9LhUoyn3g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jJ3v12tt0zd7f for ; Mon, 02 Nov 2026 07:56:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 319a0 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 02 Nov 2026 07:56:45 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Sumit Saxena From: Sumit Saxena Subject: git: cfbf11308b58 - main - if_bnxt: skip HW kTLS on old firmware and add a tunable to disable it List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: Commit messages for the main branch of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ssaxena X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: cfbf11308b586d559f7d4ce39edefe1a0c3b3742 Auto-Submitted: auto-generated Date: Mon, 02 Nov 2026 07:56:45 +0000 Message-Id: <6ae8423d.319a0.79a3941@gitrepo.freebsd.org> The branch main has been updated by ssaxena: URL: https://cgit.FreeBSD.org/src/commit/?id=cfbf11308b586d559f7d4ce39edefe1a0c3b3742 commit cfbf11308b586d559f7d4ce39edefe1a0c3b3742 Author: Sumit Saxena AuthorDate: 2026-10-24 07:22:55 +0000 Commit: Sumit Saxena CommitDate: 2026-11-02 07:55:43 +0000 if_bnxt: skip HW kTLS on old firmware and add a tunable to disable it Older Thor2 firmware serves key context allocation requests very slowly, about a second each. The driver allocates key contexts in batches of 128, so setting up 500k sessions takes roughly 3900 requests and driver load appears to hang. Enable hardware kTLS only when the firmware's HWRM interface version is 1.10.3.99 or newer. Keep the full four-part version in hwrm_spec_code_ext for the comparison. Also add a disable_hw_ktls loader tunable that turns hardware kTLS off for a device without unloading kTLS support, which helps isolate kTLS problems in the field. Reviewed by: gallatin Sponsored by: Broadcom Inc MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D60977 --- sys/dev/bnxt/bnxt_en/bnxt.h | 6 ++++++ sys/dev/bnxt/bnxt_en/bnxt_hwrm.c | 3 +++ sys/dev/bnxt/bnxt_en/bnxt_ktls.c | 14 ++++++++++++++ sys/dev/bnxt/bnxt_en/bnxt_ktls.h | 2 ++ sys/dev/bnxt/bnxt_en/bnxt_sysctl.c | 6 ++++++ sys/dev/bnxt/bnxt_en/if_bnxt.c | 3 ++- 6 files changed, 33 insertions(+), 1 deletion(-) diff --git a/sys/dev/bnxt/bnxt_en/bnxt.h b/sys/dev/bnxt/bnxt_en/bnxt.h index 5c37fb4b2857..90f624ba3b2c 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt.h +++ b/sys/dev/bnxt/bnxt_en/bnxt.h @@ -1255,6 +1255,10 @@ struct bnxt_bs_trace_info { struct bnxt_logger; +#define BNXT_HWRM_VER_CODE_EXT(maj, mnr, upd, rsv) \ + (((uint32_t)(maj) << 24) | ((uint32_t)(mnr) << 16) | \ + ((uint32_t)(upd) << 8) | (uint32_t)(rsv)) + struct bnxt_softc { device_t dev; if_ctx_t ctx; @@ -1328,6 +1332,7 @@ struct bnxt_softc { uint16_t hwrm_max_req_len; uint16_t hwrm_max_ext_req_len; uint32_t hwrm_spec_code; + uint32_t hwrm_spec_code_ext; #define BNXT_MAX_QUEUE 8 uint8_t max_tc; @@ -1598,6 +1603,7 @@ struct bnxt_softc { struct sysctl_ctx_list mpc_cmp_time_stats; struct sysctl_oid *mpc_cmp_time_stats_oid; uint32_t max_ktls_entries; + bool disable_hw_ktls; #define BNXT_CAGR_CQCOAL_OFFSET 0xc00 #define BNXT_CAGR_NQAGG_MAXTIMER 0x5930000 diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c index 54e06f3a7972..2d9983393995 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c @@ -973,6 +973,9 @@ bnxt_hwrm_ver_get(struct bnxt_softc *softc) softc->hwrm_spec_code = resp->hwrm_intf_maj_8b << 16 | resp->hwrm_intf_min_8b << 8 | resp->hwrm_intf_upd_8b; + softc->hwrm_spec_code_ext = BNXT_HWRM_VER_CODE_EXT( + resp->hwrm_intf_maj_8b, resp->hwrm_intf_min_8b, + resp->hwrm_intf_upd_8b, resp->hwrm_intf_rsvd_8b); if (resp->hwrm_intf_maj_8b < 1) { device_printf(softc->dev, "HWRM interface %d.%d.%d is older " "than 1.0.0.\n", resp->hwrm_intf_maj_8b, diff --git a/sys/dev/bnxt/bnxt_en/bnxt_ktls.c b/sys/dev/bnxt/bnxt_en/bnxt_ktls.c index 01f1c711ca0e..982292aad2a3 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_ktls.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_ktls.c @@ -305,6 +305,20 @@ void bnxt_alloc_ktls_info(struct bnxt_softc* softc, uint16_t max_keys, if (BNXT_VF(softc)) return; + if (softc->disable_hw_ktls) { + if (bootverbose) + device_printf(softc->dev, + "HW kTLS disabled by tunable\n"); + return; + } + + if (softc->hwrm_spec_code_ext < BNXT_MIN_KTLS_HWRM_VERSION) { + if (bootverbose) + device_printf(softc->dev, + "HWRM version is too low for HW kTLS\n"); + return; + } + if (!ktls) { bool partition_mode = false; struct bnxt_kctx* kctx; diff --git a/sys/dev/bnxt/bnxt_en/bnxt_ktls.h b/sys/dev/bnxt/bnxt_en/bnxt_ktls.h index a22cae2b67d0..9b48b054dd34 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_ktls.h +++ b/sys/dev/bnxt/bnxt_en/bnxt_ktls.h @@ -39,6 +39,8 @@ #include "bnxt.h" #include "bnxt_mpc.h" +#define BNXT_MIN_KTLS_HWRM_VERSION BNXT_HWRM_VER_CODE_EXT(1, 10, 3, 99) + #define BNXT_TX_CRYPTO_KEY_TYPE HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_TX #define BNXT_RX_CRYPTO_KEY_TYPE HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_RX diff --git a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c index a3d98df774f8..e7ba465d4359 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c @@ -2736,6 +2736,12 @@ bnxt_ktls_sysctls(struct bnxt_softc *softc) if (!oid) return ENOMEM; + oid = SYSCTL_ADD_BOOL(ctx, children, OID_AUTO, "disable_hw_ktls", + CTLFLAG_RDTUN, &softc->disable_hw_ktls, 0, + "Disable hardware kTLS offload on this device"); + if (!oid) + return (ENOMEM); + return 0; } diff --git a/sys/dev/bnxt/bnxt_en/if_bnxt.c b/sys/dev/bnxt/bnxt_en/if_bnxt.c index 4f380b7b956e..462cdd3015b1 100644 --- a/sys/dev/bnxt/bnxt_en/if_bnxt.c +++ b/sys/dev/bnxt/bnxt_en/if_bnxt.c @@ -2950,7 +2950,8 @@ bnxt_attach_pre(if_ctx_t ctx) if (BNXT_CHIP_P7(softc)) { rc = bnxt_ktls_sysctls(softc); if (rc) { - device_printf(softc->dev, "attach: max_ktls_entries sysctl creation failed\n"); + device_printf(softc->dev, + "attach: kTLS sysctl creation failed\n"); goto failed; }