From nobody Thu Oct 29 23:37:32 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jG0yP0XbRz6xrD9 for ; Thu, 29 Oct 2026 23:37:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jG0yN6BBjz4t6T for ; Thu, 29 Oct 2026 23:37:32 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793317052; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=aYuZJGDsfDgUmWi77afrsDayCSthXB4tL25bmMptlo0=; b=vUTT3FaFrYDPSVPP4wYERThMkN9xkwjT0E3JWKy1smoRYfwW34ROdYZqNsAiXojgGAjd/M ObF6Fku7954+/Q5NNsJlaOkaR6+KKN0AUldXMY+CssCl0P+6qaffOd0mYodHCUY31duoU9 rUCuCH7gDm7uUO2FgZBf/oVYI2NK1QYcHC7iMaSjdExXr/haQGyNoLFExdltyDI2oNXGhZ pLJj7WtzrDKKO/zKwSGYe8VP7Z3g1vsY2SPezG/c915d7PUjlyr78TrSEwVg88K9HSpGDU pXv74j8AF0Yb4O/i9qjyBx73X37jxlPagd8XvzACMKcduP9UfN3u9GqY7V6nMw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793317052; b=QQM834oG9XYvN7b7kpXcvN2QYRpKAKHBXWXztGDfk7kswxG3bYsl1JAeAVKR1qPlF0zK8x JWiCe3gwTPrOqQE7Hi3q7QsQoi2m3Z+l3LMdRF76/54UH3wnyivJ6jFvd5L5Larq7mHQWy 9W6wDXEKfjKIV7dZwGbC5ARFYzA2mxY0D84AQtAVFr2OrwFMF0zYPW2+gOMdjeaq8LMWgd lMfWql3OswV1HaHVt5nQ9DJ8H1aKXKlbQkKbsK97Rto4bicGrAX2FHpS9b8T6QFHMQFbGz McyObYIpT9Ccg/hPz/hc2OHBU4OgAAp1ATPWnceKylBkSaani3xDl1sXv2pZIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793317052; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=aYuZJGDsfDgUmWi77afrsDayCSthXB4tL25bmMptlo0=; b=t3mi4XYwi7ilULIEmXx1hXKrRmge+2Q12F0oVinPuJojTbPfbVhSMxQWMwsYRCBB3hReIz teJai4H6vrgK5xWPr/6yltXjILyTf1uusoMnEy5CdlT9b8sLT6KIdLWbSXJPdLK5RklK3l Bu1Twah+7W29wzNT7RD/vPibead7i0srWP/+WMJQXo5ggDSIVIcIIAITSpdohlq821XYme my0wmYkuIkUTQvzHEngOqP6p3wVBOeOfzwfei5oJlle75Lneif1zlZtLnax9YXPdO1AT3/ A9BRHtJJGRcl67RjqesbM5AABxg2AYeHmDsv8v4WDH/8j0gHFPdMACtbDAcB4Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jG0yN5nCRz12Hp for ; Thu, 29 Oct 2026 23:37:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 39e55 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 29 Oct 2026 23:37:32 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Cc: Sumit Saxena From: Kevin Bowling Subject: git: c6e058bea3c8 - stable/15 - if_bnxt: fix HWRM failures/timeouts after repeated FW resets List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: c6e058bea3c8f111fe2b648293a4b9c267af1fcb Auto-Submitted: auto-generated Date: Thu, 29 Oct 2026 23:37:32 +0000 Message-Id: <6ae3d8bc.39e55.5f9167@gitrepo.freebsd.org> The branch stable/15 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=c6e058bea3c8f111fe2b648293a4b9c267af1fcb commit c6e058bea3c8f111fe2b648293a4b9c267af1fcb Author: Sumit Saxena AuthorDate: 2026-07-31 16:13:44 +0000 Commit: Kevin Bowling CommitDate: 2026-10-29 23:25:51 +0000 if_bnxt: fix HWRM failures/timeouts after repeated FW resets Consecutive firmware-initiated reset cycles produced HWRM failures/timeouts and traffic didn't come back. Fix the FW-reset recovery path: - bnxt_fw_reset_close() called bnxt_stop() and bnxt_hwrm_func_drv_unrgtr(), sending HWRM ring/VNIC/filter free commands to a firmware that's already mid-reset and unresponsive, producing the observed timeouts. Both are unnecessary since firmware comes back with fresh state anyway; drop them along with the now-redundant iflib_request_reset(). - bnxt_func_reset() now skips bnxt_hwrm_resource_free() entirely while BNXT_STATE_IN_FW_RESET is set, for the same reason. - bnxt_open() (used to reopen after a firmware reset) now issues bnxt_hwrm_func_reset() up front and drives reinit through iflib's own reset machinery (iflib_request_reset() plus a new bnxt_iflib_reset_sync() that waits for IFF_DRV_RUNNING/OACTIVE to flip 1->0->1) instead of calling bnxt_init()/bnxt_intr_enable() directly. - Route repeated PORT_QSTATS/PORT_QSTATS_EXT HWRM timeouts through bnxt_log_live() instead of device_printf(), so a stats poll timing out during a reset doesn't spam dmesg on every tick. Reviewed by: gallatin Sponsored by: Broadcom Inc. Differential Revision: https://reviews.freebsd.org/D58617 (cherry picked from commit dbde459751448ab9e98662660049513db783600c) --- sys/dev/bnxt/bnxt_en/bnxt.h | 1 + sys/dev/bnxt/bnxt_en/bnxt_hwrm.c | 15 ++++++--- sys/dev/bnxt/bnxt_en/if_bnxt.c | 67 ++++++++++++++++++++++++++++++++-------- 3 files changed, 66 insertions(+), 17 deletions(-) diff --git a/sys/dev/bnxt/bnxt_en/bnxt.h b/sys/dev/bnxt/bnxt_en/bnxt.h index b23f99497310..2cd4a77d406a 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt.h +++ b/sys/dev/bnxt/bnxt_en/bnxt.h @@ -50,6 +50,7 @@ #include "bnxt_dcb.h" #include "bnxt_auxbus_compat.h" #include "bnxt_sriov.h" +#include "bnxt_log.h" #define DFLT_HWRM_CMD_TIMEOUT 500 diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c index cabfe38ced03..844f6c7ffbdc 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c @@ -256,10 +256,17 @@ _hwrm_send_message(struct bnxt_softc *softc, void *msg, uint32_t msg_len) DELAY(1000); } if (i >= softc->hwrm_cmd_timeo) { - device_printf(softc->dev, - "Timeout sending %s: (timeout: %u) seq: %d\n", - GET_HWRM_REQ_TYPE(req->req_type), softc->hwrm_cmd_timeo, - le16toh(req->seq_id)); + if (req->req_type == HWRM_PORT_QSTATS || + req->req_type == HWRM_PORT_QSTATS_EXT) + bnxt_log_live(softc, BNXT_LOGGER_L2, + "Timeout sending %s: (timeout: %u) seq: %d\n", + GET_HWRM_REQ_TYPE(req->req_type), softc->hwrm_cmd_timeo, + le16toh(req->seq_id)); + else + device_printf(softc->dev, + "Timeout sending %s: (timeout: %u) seq: %d\n", + GET_HWRM_REQ_TYPE(req->req_type), softc->hwrm_cmd_timeo, + le16toh(req->seq_id)); return ETIMEDOUT; } /* Last byte of resp contains the valid key */ diff --git a/sys/dev/bnxt/bnxt_en/if_bnxt.c b/sys/dev/bnxt/bnxt_en/if_bnxt.c index 61cbae29ecd2..0505295ac17b 100644 --- a/sys/dev/bnxt/bnxt_en/if_bnxt.c +++ b/sys/dev/bnxt/bnxt_en/if_bnxt.c @@ -1986,10 +1986,6 @@ static void bnxt_fw_reset_close(struct bnxt_softc *bp) bnxt_fw_fatal_close(bp); } - iflib_request_reset(bp->ctx); - bnxt_stop(bp->ctx); - bnxt_hwrm_func_drv_unrgtr(bp, false); - for (i = bp->nrxqsets-1; i>=0; i--) { if (BNXT_CHIP_P5_PLUS(bp)) iflib_irq_free(bp->ctx, &bp->nq_rings[i].irq); @@ -2318,10 +2314,52 @@ static bool bnxt_fw_reset_timeout(struct bnxt_softc *bp) (bp->fw_reset_max_dsecs * HZ / 10)); } +/* enqueue an iflib reset and wait for RUNNING/OACTIVE to flip 1->0->1. */ +static int +bnxt_iflib_reset_sync(struct bnxt_softc *bp, int timeout_us) +{ + unsigned flags; + bool reset_started; + int remaining_us = timeout_us; + + /* request reset (async) and enqueue admin task */ + iflib_request_reset(bp->ctx); + iflib_admin_intr_deferred(bp->ctx); + + /* see if reset already started */ + flags = if_getdrvflags(bp->ifp); + reset_started = ((flags & IFF_DRV_RUNNING) == 0) || + ((flags & IFF_DRV_OACTIVE) != 0); + + while (remaining_us > 0) { + flags = if_getdrvflags(bp->ifp); + + if (!reset_started) { + /* wait for reset to START (RUNNING=0 or OACTIVE=1) */ + if ((flags & IFF_DRV_RUNNING) == 0 || (flags & IFF_DRV_OACTIVE) != 0) + reset_started = true; + } else { + /* wait for reset to FINISH (RUNNING=1 and OACTIVE=0) */ + if ((flags & IFF_DRV_RUNNING) != 0 && (flags & IFF_DRV_OACTIVE) == 0) + return 0; /* success */ + } + + pause_sbt("bnxtrst", SBT_1US * 20, 0, 0); /* 20 us sleep */ + remaining_us -= 20; + } + return ETIMEDOUT; +} + static int bnxt_open(struct bnxt_softc *bp) { int rc = 0; + rc = bnxt_hwrm_func_reset(bp); + if (rc) { + device_printf(bp->dev, "reinit: hwrm func_reset failed\n"); + return rc; + } + /* A function/chip reset drops the CAGR tick-resolution register * programmed in bnxt_attach_pre(); restore it before we start * relying on interrupt coalescing again. */ @@ -2388,10 +2426,6 @@ static int bnxt_open(struct bnxt_softc *bp) } bnxt_msix_intr_assign(bp->ctx, 0); - rc = bnxt_init_hw(bp->ctx); - if (rc != 0) - return (rc); - bnxt_intr_enable(bp->ctx); if (test_and_clear_bit(BNXT_STATE_FW_RESET_DET, &bp->state)) { if (!test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) { @@ -2399,7 +2433,13 @@ static int bnxt_open(struct bnxt_softc *bp) } } - device_printf(bp->dev, "Network interface is UP and operational\n"); + rc = bnxt_iflib_reset_sync(bp, 3000000); + if (rc) { + device_printf(bp->dev, "timeout waiting for reset/init\n"); + return rc; + } + + device_printf(bp->dev, "FW Reset Successfull, Interface is UP and Operational\n"); return rc; } @@ -2496,9 +2536,8 @@ static void bnxt_fw_reset_task(struct work_struct *work) case BNXT_FW_RESET_STATE_OPENING: rc = bnxt_open(bp); if (rc) { - device_printf(bp->dev, "bnxt_open() failed during FW reset\n"); + device_printf(bp->dev, "Firmware reset failed (err=%d)\n", rc); bnxt_fw_reset_abort(bp, rc); - rtnl_unlock(); return; } @@ -3460,7 +3499,8 @@ bnxt_func_reset(struct bnxt_softc *softc) return; } - bnxt_hwrm_resource_free(softc); + if (!test_bit(BNXT_STATE_IN_FW_RESET, &softc->state)) + bnxt_hwrm_resource_free(softc); return; } @@ -5748,7 +5788,8 @@ bnxt_handle_async_event(struct bnxt_softc *softc, struct cmpl_base *cmpl) softc->fw_health->survivals++; set_bit(BNXT_STATE_FW_NON_FATAL_COND, &softc->state); } - device_printf(softc->dev, + device_printf(softc->dev, "%s firmware reset event arrived.\n", type_str); + bnxt_log_live(softc, BNXT_LOGGER_L2, "%s firmware reset event, data1: 0x%x, data2: 0x%x, min wait %u ms, max wait %u ms\n", type_str, data1, data2, softc->fw_reset_min_dsecs * 100,