From nobody Fri Nov 06 07:33:25 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jLWBF6Rx3z6yyNK for ; Fri, 06 Nov 2026 07:33:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jLWBF59Dsz4h7W for ; Fri, 06 Nov 2026 07:33:25 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793950405; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jCxju5FWP8Muix41lRG83REKTby0haLWz+MJAaF0rbY=; b=rn2Vei7pyl/rCHS83ut0m6vCtA15xY7a6tbjCE02kWELc8RQEb1OdnrAvkbffvJOsQBAx8 CxN+QA60w53HK/bxzhyflDdGfunHVARsQzBU1jf20mDTzwCPWjzl+EpSP97JHSTvXCKNVZ BwKJA334HJS1agFZETTHwjldmjWJXBOEXuUZdely+FcDgcBuhyPQzXcn28rceKQd9QUxBw 6rxrNNjHFaSAh+jik784Qg7VkL32DQiwccpcd920IzldwCdA2583gLJgZMSf1up96k5pdY JMAFBITpjCJyGUibKiE6xmmUkl6LlJBvQOcb3Nn41P84o3ej4zQInadug9wdng== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793950405; b=kdGxHtl8jE+pMNSnhOjIK6BRhmoY99GxC/HhR+jq2wQIhH/fDSlZGD/PYCmITjJYNaAVzX sXHY2vU50ipnqYGAaKXfyhG53acaBM3E3ojGTuZ/59PqXVmQPGpYijopiskW0QQQiDsw75 n848DKxLUJ/Y9WLer4FyReyd4WY0qifza4l/LJV6tBMGLeccy8OFpLd7Ii8MMr1hIAZdvt F8qtZfOGY2jeJnuRnaT6D31ZUVJm670yKySkwnkq6YdCcMQn7TgcuD56t0vGNcD9QjxSbh If11bH6Zg58z8w8gaaUv03avQ8um1ZX+CoyuiTe1vv0dnUw/RPfdvuXgMpwruA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793950405; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jCxju5FWP8Muix41lRG83REKTby0haLWz+MJAaF0rbY=; b=nW9l+ZyOgbnUzFhn86Zp5hNzgWQhGRZjKxjFv6qca1uFL2IqkwTqKiEpvn63pe42MA6zzb qTkNKJF/xdwaTa+dht+cjdmPZv1QaAPtXrp9ldHxS1Tg00p8O7VLS8DB97/C4s2K8uQOZq UCdw1Qp/RpoYI+XE8+9E7DpUIWg554iJSwDdiXVpYvY9jLBmlEFR15oYeiYfeXOL9IMIFw 1lUBv6zx/tS0zQ5wyjcb5zznQMl6UJJEG9xcC8+hOne8zTepvzNtc/SUFawcfKF3PaoA7y PeHPnUU0KyPWZkzo9tjLqg8728YKX4IfiFNbSGzvMhpPsZLTGCuQTZm2rbOB9g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jLWBF4p1jzR7d for ; Fri, 06 Nov 2026 07:33:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1de37 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 06 Nov 2026 07:33:25 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Cc: Sumit Saxena From: Sumit Saxena Subject: git: 1dd8d2d8173e - stable/15 - if_bnxt: skip HW kTLS on old firmware and add a tunable to disable it List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ssaxena X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 1dd8d2d8173eff0fc969b4f858739c302ad27768 Auto-Submitted: auto-generated Date: Fri, 06 Nov 2026 07:33:25 +0000 Message-Id: <6aed82c5.1de37.76dc9538@gitrepo.freebsd.org> The branch stable/15 has been updated by ssaxena: URL: https://cgit.FreeBSD.org/src/commit/?id=1dd8d2d8173eff0fc969b4f858739c302ad27768 commit 1dd8d2d8173eff0fc969b4f858739c302ad27768 Author: Sumit Saxena AuthorDate: 2026-10-24 07:22:55 +0000 Commit: Sumit Saxena CommitDate: 2026-11-06 07:33:03 +0000 if_bnxt: skip HW kTLS on old firmware and add a tunable to disable it Older Thor2 firmware serves key context allocation requests very slowly, about a second each. The driver allocates key contexts in batches of 128, so setting up 500k sessions takes roughly 3900 requests and driver load appears to hang. Enable hardware kTLS only when the firmware's HWRM interface version is 1.10.3.99 or newer. Keep the full four-part version in hwrm_spec_code_ext for the comparison. Also add a disable_hw_ktls loader tunable that turns hardware kTLS off for a device without unloading kTLS support, which helps isolate kTLS problems in the field. Reviewed by: gallatin Sponsored by: Broadcom Inc MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D60977 (cherry picked from commit cfbf11308b586d559f7d4ce39edefe1a0c3b3742) --- sys/dev/bnxt/bnxt_en/bnxt.h | 6 ++++++ sys/dev/bnxt/bnxt_en/bnxt_hwrm.c | 3 +++ sys/dev/bnxt/bnxt_en/bnxt_ktls.c | 14 ++++++++++++++ sys/dev/bnxt/bnxt_en/bnxt_ktls.h | 2 ++ sys/dev/bnxt/bnxt_en/bnxt_sysctl.c | 6 ++++++ sys/dev/bnxt/bnxt_en/if_bnxt.c | 3 ++- 6 files changed, 33 insertions(+), 1 deletion(-) diff --git a/sys/dev/bnxt/bnxt_en/bnxt.h b/sys/dev/bnxt/bnxt_en/bnxt.h index 971c73d67dd0..e21c8c26945e 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt.h +++ b/sys/dev/bnxt/bnxt_en/bnxt.h @@ -1255,6 +1255,10 @@ struct bnxt_bs_trace_info { struct bnxt_logger; +#define BNXT_HWRM_VER_CODE_EXT(maj, mnr, upd, rsv) \ + (((uint32_t)(maj) << 24) | ((uint32_t)(mnr) << 16) | \ + ((uint32_t)(upd) << 8) | (uint32_t)(rsv)) + struct bnxt_softc { device_t dev; if_ctx_t ctx; @@ -1328,6 +1332,7 @@ struct bnxt_softc { uint16_t hwrm_max_req_len; uint16_t hwrm_max_ext_req_len; uint32_t hwrm_spec_code; + uint32_t hwrm_spec_code_ext; #define BNXT_MAX_QUEUE 8 uint8_t max_tc; @@ -1598,6 +1603,7 @@ struct bnxt_softc { struct sysctl_ctx_list mpc_cmp_time_stats; struct sysctl_oid *mpc_cmp_time_stats_oid; uint32_t max_ktls_entries; + bool disable_hw_ktls; #define BNXT_CAGR_CQCOAL_OFFSET 0xc00 #define BNXT_CAGR_NQAGG_MAXTIMER 0x5930000 diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c index 795797a34b85..2db73f607c0f 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c @@ -973,6 +973,9 @@ bnxt_hwrm_ver_get(struct bnxt_softc *softc) softc->hwrm_spec_code = resp->hwrm_intf_maj_8b << 16 | resp->hwrm_intf_min_8b << 8 | resp->hwrm_intf_upd_8b; + softc->hwrm_spec_code_ext = BNXT_HWRM_VER_CODE_EXT( + resp->hwrm_intf_maj_8b, resp->hwrm_intf_min_8b, + resp->hwrm_intf_upd_8b, resp->hwrm_intf_rsvd_8b); if (resp->hwrm_intf_maj_8b < 1) { device_printf(softc->dev, "HWRM interface %d.%d.%d is older " "than 1.0.0.\n", resp->hwrm_intf_maj_8b, diff --git a/sys/dev/bnxt/bnxt_en/bnxt_ktls.c b/sys/dev/bnxt/bnxt_en/bnxt_ktls.c index 01f1c711ca0e..982292aad2a3 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_ktls.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_ktls.c @@ -305,6 +305,20 @@ void bnxt_alloc_ktls_info(struct bnxt_softc* softc, uint16_t max_keys, if (BNXT_VF(softc)) return; + if (softc->disable_hw_ktls) { + if (bootverbose) + device_printf(softc->dev, + "HW kTLS disabled by tunable\n"); + return; + } + + if (softc->hwrm_spec_code_ext < BNXT_MIN_KTLS_HWRM_VERSION) { + if (bootverbose) + device_printf(softc->dev, + "HWRM version is too low for HW kTLS\n"); + return; + } + if (!ktls) { bool partition_mode = false; struct bnxt_kctx* kctx; diff --git a/sys/dev/bnxt/bnxt_en/bnxt_ktls.h b/sys/dev/bnxt/bnxt_en/bnxt_ktls.h index a22cae2b67d0..9b48b054dd34 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_ktls.h +++ b/sys/dev/bnxt/bnxt_en/bnxt_ktls.h @@ -39,6 +39,8 @@ #include "bnxt.h" #include "bnxt_mpc.h" +#define BNXT_MIN_KTLS_HWRM_VERSION BNXT_HWRM_VER_CODE_EXT(1, 10, 3, 99) + #define BNXT_TX_CRYPTO_KEY_TYPE HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_TX #define BNXT_RX_CRYPTO_KEY_TYPE HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_RX diff --git a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c index a3d98df774f8..e7ba465d4359 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c @@ -2736,6 +2736,12 @@ bnxt_ktls_sysctls(struct bnxt_softc *softc) if (!oid) return ENOMEM; + oid = SYSCTL_ADD_BOOL(ctx, children, OID_AUTO, "disable_hw_ktls", + CTLFLAG_RDTUN, &softc->disable_hw_ktls, 0, + "Disable hardware kTLS offload on this device"); + if (!oid) + return (ENOMEM); + return 0; } diff --git a/sys/dev/bnxt/bnxt_en/if_bnxt.c b/sys/dev/bnxt/bnxt_en/if_bnxt.c index 4f380b7b956e..462cdd3015b1 100644 --- a/sys/dev/bnxt/bnxt_en/if_bnxt.c +++ b/sys/dev/bnxt/bnxt_en/if_bnxt.c @@ -2950,7 +2950,8 @@ bnxt_attach_pre(if_ctx_t ctx) if (BNXT_CHIP_P7(softc)) { rc = bnxt_ktls_sysctls(softc); if (rc) { - device_printf(softc->dev, "attach: max_ktls_entries sysctl creation failed\n"); + device_printf(softc->dev, + "attach: kTLS sysctl creation failed\n"); goto failed; }